Dbmonpreflightcheck File Is Found Locally

CUCMuses IBM Informix for database needs. Nosotros have no native access to this (unless you lot are a Cisco TAC Engineer who tin can gain temporary access to root).

If DB replication breaks, nosotros see many symptoms in our IPT network like Phone registered to a Subscriber unable to brand calls to phones registered on the other subscriber, unable to login to extension mobility etc.

We can make employ on any one of these methods to check replication status:

•  Open up RTMT -- Click on : Telephone call Manager --> Service --> Database Summary

• Open up a web browser, type in https://<IP accost>:8443/cucreports/ ,Enter your authorized username and password.
  Go to : Database Condition Report

• Using putty, SSH to the CUCM to have CLI access and run this control : utils dbreplication runtimestate --> REPl.LOOP? is the current state.

Here is what the replication state ways :
0 - Initialization state : This country indicates that replication is in the process of trying to  setup. Being in this country for a menstruum longer than an hour could  indicate a failure in setup.

1 - Number of replicates not correct : This state is rarely seen, can signal its notwithstanding in the setup process. Existence in this state for a period longer than an 60 minutes could indicate a failure in setup.

2 - Replication is good : All is well in paradise :)

3 - Tables are suspect : Logical connections have been established but we are unsure if tables match. This can happen because the other servers are unsure if  there is an update to a user facing feature that has not been passed  from that sub to the other device in the cluster.

four - Setup failed / Dropped : The server no longer has an active logical connectedness to receive  database table across. No replication is occurring in this state.

If State is other than 2, cheque:

• Server & Cluster connectivity : Check TCP/UDP ports needed to be opened on the network. To get the port list for your CUCM version, but google : CUCM < your version> ports

• Configuration files (if on older CUCM - extremely rare) :

1. /etc/hosts ---> local resolution of hostnames to IP addresses
2. /home/informix/.rhosts ---> hosts trusted to make database connections
3. $INFORMIXDIR/etc/sqlhosts ---> full list of CCM servers for replication

• Check if server times are correct and synced (NTP working fine)
• DNS not configured properly (forward/opposite lookup)
• NTP non reachable/time migrate between server
• A Cisco DB replicator service not running/non working
• Cisco Database Layer monitor (DbMon) hung/stopped

Useful Commands :

• utils dbreplication setrepltimeout -- The default value is fix to 300 seconds. Y'all can validate this by running "bear witness tech repltimeout". This is the timer used to put multiple servers into one run of the information sync. In other words, it is the "batching" timer. This affects when the circulate realize template and data sync will fire (n seconds from the terminate of the first defined server). Clustering over WAN (Moo-cow) long delays can cause the data sync process to be exponentially longer. Effort to sync the local servers first.

• utils dbreplication repair -- in CUCM five.10, this command meant a reset of the replication, whereas, in CUCM 6.x and college versions, this ways a repair of the data. It runs a repair process on all tables in the replication for all servers that are included in the command. Run this control when RTMT = ii, not when RTMT = 0 or 3.

• utils dbreplication repairtable / repairreplicate -- This command substantially does the same affair every bit the repair command, just runs on only one table / replicate, hence making the process much faster. It fixes the out of sync data for that table / replicate. Yous can verify past running "utils dbreplication condition" to see if there are any mismatches or errors found. It is especially useful on large CUCM clusters. Run this command when RTMT = 2, not when RTMT = 0 or 3.

• utils dbreplication stop -- Y'all should simply be running this if y'all want to stop replication setup. The only way to recover from a stop is with a reset. This command removes the set-up indicator file i.e. the dbmonpreflightcheck file and kills the currently running replication commands. It pauses for the duration of repltimeout timer, so if you lot run replication commands shortly after running a stop, it could kill the commands again. Run this command when RTMT = 0, not when RTMT = 3

• utils dbreplication reset -- This command causes replication to exist torn downwardly and then set-up. Y'all should run this command when RTMT = four or when yous have issued stop. Successful completion of this process results in RTMT = 2.

• utils dbreplication clusterreset -- Avoid running this command. It is for debugging replication set-upwards problems. It bypasses the RTMT settings, cluster requirements and normal CUCM set-up. It causes services to get out of sync with the database considering information technology syncs data without change notification. The services need to be restarted when this command is run, no exceptions!

• utils dbreplication dropadmindb -- Run this command when there is a looping effort to define a server in replication. It'southward usually not the server that'southward failing, it'south the pub which is corrupted as a issue of an attempt or the sub, prior to the electric current ane attempting set-up.

• utils dbreplication forcedatasyncsub -- This command takes a backup of the publisher and restores information technology to the subscriber(s) and resets upwardly replication. It requires a serivces restart on the subscriber so they get the new values.

New Commands and Database Improvements in CUCM nine.x:

• Re-engineered CLI forcedatasyncsuball (Lightening fast) -- This command tin now restore a larger cluster in a shorter flow of fourth dimension!

• New CLI rebuild is a stop, drop and reset all in one (and faster) -- The architecture of Rebuild is multi-threaded, the total functioning time is much shorter than executing iii different CLI commands (end / driblet / reset). Rebuild, is a primary command that will stop, delete and trigger the replication setup signal beyond the cluster automatically and in parallel:

1. Stop DB Replication – finish the electric current replication setup process if exists

1. Remove server from database – Remove replication from the network by either "cdr delete", dropping the syscdr database or renaming the syscdr database remotely

1. Trigger Dbmon on the subscriber to submit a replication setup asking on to publisher.

• New CLI utils replication status table/replicate -- The "utils dbreplication status" control is lengthy when it runs. If only one table is suspect, and then y'all have to wait for all the tables to bank check. Being able to check one table speeds upward checking of replication.

• Better Log Collection -- "utils create written report database" collects all the database logs in one go. As well, ercollect.sh script is embedded into the server for IBM root cause cases. The script is on the server now, no demand to transfer and change permissions. It is accessible via root access merely.

• Faster and more than accurate Runtimestate CLI -- This command is now multithreaded, making it much faster. The output will besides be logged for historical RCA. If there are whatever unreachable servers in the cluster, this command volition no longer hang. Some boosted information will be included in it such every bit repltimeout and IDS server number.

- Abhinay Mylavarapu

schaferfaut1952.blogspot.com

Source: http://voipholic.blogspot.com/2014/03/troubleshooting-cucm-database.html

Share this post