DBAmon Error Messages

Event/Error Message Reference

Error Message Format

Example: DBA210W

 DBA - Common to all messages
 210 - Unique message identifier (see below)
 W   - Message severity
               E: DBAmon program error
               N: DBAmon program information message
               C: DBAmon critical event
               W: DBAmon warning event
               I: DBAmon informational event
               I: DBAmon user event

DBA005 - "DBAmon Governor Is Active"

Description: Due to a high ticket rate, the DBAmon Governor has activated itself. See Governor Overview for more information.
Corrective Action: DBAmon will automatically inhibit monitoring of all instances for 1-2 hours. Once the problem is solved, you can manually remove the contents of the ALL inhibit file to resume monitoring, but the situation will correct itself.

DBA010 - "Logic Error - Invoked dbamon_orant.pl Did Not Finish"

Description: While attempting to check Oracle on NT, the DBAmon probe script failed. Check accompanying error message from "D" WWW page or DBAmon event email.
Corrective Action: Solve problem indicated by event diagnostics.

DBA020 - "Error(s) Found in DBC File: $dbcfilename[$i]"

Description: You have specified a DBC parameter which does not exist.
Corrective Action: The reference of correct DBC parameters can be could at DBC Reference. Specfify a valid DBC parameter.

DBA101 - "$applhost - Missing $F3 process(es) - Number Found: $F5 - Should Be: $F6"

Description: SAP/UX - There are missing SAP work processes of the type indicated.
Corrective Action:

DBA101 - "$applhost - SAP Down - sapstart Process Not Running - Number Found: $F5 - Should Be: $F6"

Description: DBAmon did not find the sapstart process running. It should be if SAP is up.
Corrective Action: Start SAP and ensure the sapstart process is running.

DBA102 - "$applhost - SAP (R3check) Cannot Connect to DB"

Description: DBAmon ran R3check to test the connection to the DB and it was not able connect.
Corrective Action: See the accompanying messages for details. Solve problem so that R3check -d works.

DBA165 - "$applhost - /usr/sap/SID Filesystem is $bdfpct full ($bdfaval MB Free)"

Description: The named filesystem is >= 99% full.
Corrective Action: Remove unneeded files or resize filesystem.

DBA166 - "$applhost - Error running bdf /usr/sap/SID"

Description: The bdfcmd to check filesystem full failed.
Corrective Action: Check the accompanying messages and act accordingly.

DBA209 - "No onbar/ontape archive data retrieved (No backup ever or DBAmon pgm. error)"

Description: It was detected by the absence of any onarchive or onbar/ontape archive data that no database archive has ever been run for this instance.
Corrective Action: Run on onbar/ontape level 0 backup.

DBA210 - "(Backup Not Running) $rounded1-$rounded2 Hours since last good onarchive backup (threshold=$backup_age[$thishost])"

Description: This event is generated when the age of the most recent onarchive backup for an Informix instance exceeds the Backup_Age specified for this host in dbamonrc.
Corrective Action: Run on onarchive backup.

DBA211 - "$rounded1-$rounded2 Hours since last good onbar/ontape backup (threshold=$backup_age[$thishost])"

Description: This event is generated when the age of the most recent onbar/ontape backup for an Informix instance exceeds the Backup_Age specified for this host in dbamonrc.
Corrective Action: Run a backup.

A system (possibly for Y2K testing) has its date set to a date in the future.
Backup(s) are completed.
The date is set back to the current date.

Corrective Action: From the error message, determine which backup (L0 or L1) occured in the future. Rerun that backup.

DBA220 - "Critical Informix Message Log Message(s) Found"

Description: After issuing the onstat -m command, one (or more) of the following strings were found (the strings are specified in /opt/dbamon/adm/dbamon.msg_critical):

Full
Error
Fail Consistency Check
failed
PANIC

Corrective Action: Research the cause of the message and any accompanying onstat -m messages.

DBA221 - "DBS Reporting Error: $error_text"

Description: After issuing the onstat -m command, one (or more) of the following strings were found:

dynamically allocated new shared memory segment

Corrective Action: Research the cause of the message and any accompanying onstat -m messages.

DBA222 - "TEMP DBSpace Full - DBS Probe Error"

Description: DBAmon was attempting to see how full your dbspaces are, but it encountered an error because the TEMP dbspace is full.
Corrective Action: Solve the TEMP space problem and allow DBAmon to rerun the probe.

DBA223 - "Informational Informix Message Log Message(s) Found"

Description: DBAmon looks in the Informix message log for all strings found int /opt/oracle/adm/dbamon.msg_warn. If it finds one or more of the strings, this event occurs.
Corrective Action: Solve the Informix problem causing this message(s).

DBA230 - "Total llog files=$num_llogs Full llog files=$full_llogs ($logpct percent)"

Description: From onstat -l command output, it was determined that > 60% of the log files are full.
Corrective Action: If this system archive logs (onconfig LTAPEDEV not equal /dev/null) determine why log archive task is not executing. For systems where LTAPEDEV is /dev/null, this is probably indicative of a long transaction (see Informix System Admin guide for info on long transactions).

DBA231 - "Fatal dbaccess Error Checking Logical Logs"

Description: Check accompanying messages.
Corrective Action: Solve problem.

DBA232 - "Fatal dbaccess Error Checking DBSpaces"

Description: Check accompanying messages.
Corrective Action: Solve problem.

DBA233 - "Fatal dbaccess Error Checking Locks"

Description: Check accompanying messages.
Corrective Action: Solve problem.

DBA234 - "Fatal Error running onstat -"

Description: Check accompanying messages.
Corrective Action: Solve problem.

DBA236 - "$F2 (DBC Parm) File=$F3 Not Found -or- Not Readable"

Description: DBAmon was looking for the filename mentioned here because of the DBC parameters that you specifed. The file was not found or DBAmon lacks permission to open it.
Corrective Action: Solve problem.

DBA237 - "SQL Will Not Run"

Description: DBAmon tried to select from sysdatabase to see if Informix is up. That SQL statement failed.
Corrective Action: Solve problem.

DBA238 - "Informix Object(s) Found Whose Size Exceeds 25gB"

Description: There is a hard limitation (as of 7.x) that no table (or table fragment) may be 32gB in size. The number isn't exactly 32gB and it depends on a number of factors (number of indices, ...), but it is approximately 32gB. The objects listed in this event are at least 25gB in size.
Corrective Action: Reduce the size of these objects, or if that is not possible, fragment the objects.

DBA239 - "ROOTPATH: $F2 Device File Is Not Readable"

Description: In order to properly monitor your Informix DB, the DBAmon probe must be able to read the header pages of the first root chunk. This event means that DBAmon does not have UX read-access to the device file for the first root chunk.
Corrective Action: The problem may be the the Userid: DBC parameter is not equal to the userid that this Informix instance is running with. You must specify a UX userid in the Userid: DBC parameter which can read the first root chunk.

DBA240 - "Read Hit Ratio $rh < Threshold of $t_readhit[$thishost]"

Description: From onstat -p output, the %cached (reads) is less than the T_Read_Hit.
Corrective Action: The standard remedy for this is to increase the BUFFERS onconfig parameter. However, some applications will always have a small write hit ratio (due to large row sizes). The general rule of thumb is to increase the size of BUFFERS until you reach a diminishing return.

DBA241 - "Write Hit Ratio $wh < Threshold of $t_writehit[$thishost]"

Description: From onstat -p output, the %cached (writes) is less than the T_Write_Hit.
Corrective Action: The standard remedy for this is to increase the BUFFERS onconfig parameter. However, some applications will always have a small write hit ratio (due to large row sizes). The general rule of thumb is to increase the size of BUFFERS until you reach a diminishing return.

DBA250 - "DBSpace $dbspace (${dbspct}% full ${dbsfree}MB free) exceeds critical threshold of $t_diskcrit[$thishost]"

Description: From the onstat -d command, it was determined that one or more dbspaces exceed the T_Disk_Full dbamonrc critical value.
Corrective Action: If it is not possible to remove any data (tempdbs), add a chunk of space to the dbspace.

DBA251 - "DBSpace $dbspace (${dbspct}% full ${dbsfree}MB free) exceeds warning threshold of $t_diskwarn[$thishost]"

Description: From the onstat -d command, it was determined that one or more dbspaces exceed the T_Disk_Full dbamonrc warning value.
Corrective Action: If it is not possible to remove any data (tempdbs), add a chunk of space to the dbspace.

DBA260 - "Informix Instance Not On-Line/Read-Only"

Description: An instance which has been designated Must_be_up: = y in dbamonrc in not On-Line. DBAmon runs the onstat -i command to check the status of an Informix instance.
Corrective Action: Inspect Informix log and bring system to On-Line (multiuser) mode.

DBA261 - "Informix Instance Not On-Line/Read-Only"

Description: An instance which has NOT been designated Must_be_up: = y in dbamonrc in not On-Line. DBAmon runs the onstat -i command to check the status of an Informix instance.
Corrective Action: Inspect Informix log and bring system to On-Line (multiuser) mode.

DBA265 - "INFORMIXDIR Filesystem is $bdfpct full ($bdfaval MB Free)."

Description: The filesystem that INFORMIXDIR resides on is >= 99% full.
Corrective Action: Remove unneeded files, or expand the filesystem.

DBA266 - "Error running 'bdf \$INFORMIXDIR'"

Description: While attempting to check the filesystem that INFORMIXDIR resides on, there was an error.
Corrective Action: Solve the problem from the diagnostic messages that were displayed with this error.

DBA270 - "Table(s) found with >= 200 extents"

Description: The tables listed in this message are in more than 200 extents. There is an Informix limitation of ~219 extents per table. If a table reaches this limit, it will not be able to grow.
Corrective Action: Reorganize the table with a large extent size.
Primary Support Action: Inform the customer of the situation and the need to reorg table(s) in this state.

DBA271 - "Table(s) found with >= $max_extents[$thishost] extents"

Description: The tables listed in this message are in more extents than what you specified in the dbamonrc parm Max_Extents. There is an Informix limitation of ~219 extents per table. If a table reaches this limit, it will not be able to grow.
Corrective Action: Reorganize the table with a large extent size.
Primary Support Action: Inform the customer of the situation and the need to reorg table(s) in this state.

DBA280 - "$offline_chunks Offline Chunk(s) Found"

Description: From onstat -d, it was determined that there are chunks that are NOT in a PO/MO state.
Corrective Action: This usually means that some kind of disk hardware error has occured. Check the Informix log for messages indicating the error that caused the chunks to go offline.

DBA290 - "Connect Failure: ($conshort) After $remsh_try Attempt(s) - Server $pingable - Err=($remsherr) RC=$rc"

Description: DBAmon uses remsh to execute commands on all systems. This error means that the remsh command failed.
Corrective Action: This command can mean that the system in question is down (UX is down). It can also indicate network problems between the system running DBAmon and this system.

DBA291 - "Downloaded DBAmon Probe Software Not Found - Will retry download on next iteration"

Description: DBAmon expected the DBAmon probe software to be on this server, but it was gone. Possible causes:

Someone deleted the software from the server
For windows servers, perhaps the default directory location is something other than \. It must be the root dir of any drive.

Corrective Action: Find out why it was deleted. For Windows servers, check to make sure that the default dir of the remsh service is \.

DBA292 - "Connect resulted in compilation error"

Description: When DBAmon tried to run a problem on the server corresponding to this instance, a PERL compilation error resulted.
Corrective Action: Probably a DBAmon bug, or incorrect installation (does /usr/local/bin/perl point to a valid version of perl5?).

DBA293 - "Probe Connect TIMEOUT (dbamonrc:Probe_Timeout=$probe_timeout)"

Description: This event will only occur if you have specifed dbamonrc Probe_Timeout: parameter. The number of seconds that the connection to this instance took exceeds the Probe_Timeout: value. The connection was killed.
Corrective Action: The DB instance or server are probably hung. Respond accordingly.

DBA299 - "Informix BUG Encountered (lockspct > 100) - Used Locks=$locksused Max Locks=$locksmax ($lockspct percent)"

Description: There is a bug in some versions of Informix 7.30 where the LOCKS column of the syssesprof SMI table has incorrect values. That must be the case here.
Corrective Action: Ignore - This bug will (someday) be fixed by Informix.

DBA300 - "Used Locks=$locksused Max Locks=$locksmax ($lockspct percent)"

Description: This instance of Informix is in danger of using all available LOCKS
Corrective Action: If possible (due to shared memory constraints), increase the LOCKS ONCONFIG parameter (Informix restart required).

DBA301 - "ALL - DBAmon checking inhibited Date=$nowdate Hour=$nowhour"

Description: Monitoring for this instance has been inhibited.
Corrective Action:

DBA301 - "DBAmon checking inhibited Day=$nowday Monitor_Days=$mon_days[$h]"

Description: Monitoring for this instance has been inhibited.
Corrective Action:

DBA301 - "DBAmon checking inhibited DayHour=$dayhour Monitor_Excl=$mon_excl[$h]"

Description: Monitoring for this instance has been inhibited.
Corrective Action:

DBA301 - "DBAmon checking inhibited Hour=$nowhour Monitor_Hours=$mon_hours[$h]"

Description: Monitoring for this instance has been inhibited.
Corrective Action:

DBA301 - "DBAmon checking inhibited by ora_dbshut"

Description: Monitoring for this instance has been inhibited.
Corrective Action:

DBA301 - "This Server - DBAmon checking inhibited Date=$nowdate Hour=$nowhour"

Description: Monitoring for this instance has been inhibited.
Corrective Action:

DBA301 - "This Server_Instance - DBAmon checking inhibited Date=$nowdate Hour=$nowhour"

Description: Monitoring for this instance has been inhibited.
Corrective Action:

DBA302 - "FILECHECK File=$fc_file Not Found"

Description: You have specifed the FILECHECK parameter in the /opt/dbamon/bin/download/dbamon_[inf|ora].cfg file. The filename that you specified is not found.
Corrective Action: Place the file on the DB server.

DBA303 - "FILECHECK File=$fc_file Is > 28 Hours Old"

Description: You have specifed the FILECHECK parameter in the /opt/dbamon/bin/download/dbamon_[inf|ora].cfg file. This means that > 28 hours have elapsed since this file was last updated.
Corrective Action: Update the file on the DB server.

DBA304 - "DBAmon checking inhibited by AUTO-INHIBIT"

Description: Some processes (like standby refresh tools) bring down a database regularly. DBAmon allows tools to create a file called /tmp/DBAmon_Lock_{ORACLE_SID}.txt which will prevent DBAmon from monitoring. This event means that this file was found to exist.
Corrective Action: If the tool which touched the lock file removes the lock file, then this event will correct itself.

DBA305 - "There are $userpw_cnt Non-System USERs whose userid=password"

Description: Some Non-System Oracle users were found with the userid equal to the password. This is a breach of security.
Corrective Action: Alter the password for these users to an unpredictable value.

DBA306 - "There are $userpwchanged_cnt System USERs whose userid=password - password changed"

Description: Some System Oracle users were found with predicable passwords. They were changed to the value that you specified.
Corrective Action: None.

DBA306 - "There are $userpwsys_cnt System USERs whose userid=password"

Description: Some System Oracle users were found with obvious passwords. This is a breach of security.
Corrective Action: Alter the password for these users to an unpredictable value.

DBA308 - "DB is in ARCHIVELOG mode, but the archiver is STOPPED"

Description: The database is "doomed" because it is in archivelog mode, but the archiver is STOPPED. The online redo logs will fill and hang the instance. If you run orastat -l you should see that "Automatic Archival" is disabled.
Corrective Action:

To correct this dynamically, ALTER SYSTEM ARCHIVE LOG START .
To correct this permanently, in init.ora change log_archive_start to true.

DBA309 - "Instance is HUNG waiting for ARCHIVER process to finish (All ONLINE REDO LOGS are full)"

Description: The DB is hung and requires immediate attention. The online redo logs all need to be archived and for some reason the ARCHIVER is not clearing them out. THIS IS NOT A BACKUP PROBLEM. It is an archiver problem.
Corrective Action: See if the archiver is running. Run archive log list and look at the Automatic archival line. If it is Disabled, then you need to start the archiver (see DBA308).

DBA310 - "Instance has $shmvseg[$thishost] V shmem segments (critical threshold=$shmcrit)"

Description: This Informix instance has > $shmcrit shmem segments. This can be bad for performance.
Corrective Action: Consolidate the total size of all segments into a smaller number of larger segments.

DBA311 - "Instance has $shmvseg[$thishost] V shmem segments (warning threshold=$shmcrit)"

DBA312 - "There are $dbarole_cnt Non-System USERs who have been granted the DBA role"

Description: The Oracle ID's listed have been granted the DBA role. They therefore that the authority to do nasty things like stopping Oracle, changing the SYS/SYSTEM password, etc..
Corrective Action: If desired, revoke DBA from the user. First, ensure the ORADBA has been granted to the user. Then revoke DBA from the user.

DBA313 - "There are $znum Possible Orphan Datafile(s) - $orphan_size gB (Threshold: 14 Days Since Last Update)"

Description: DBAmon was looking for datafiles that once belonged to this instance which are no longer mentioned in any data dictionary table. In order to qualify, the datafile must not have been touched for the last 14 days.
Corrective Action: "rm" or gzip these datafiles.

DBA314 - "There are $sec_fileperm_cnt[$thishost] Oracle file(s) with incorrect permission(s)"

Description: An Oracle file has file permission that violates security policy.
Corrective Action: chmod the file(s) to the proper permission.

DBA315 - "There are $sec_dbarole_cnt[$thishost] Non-System USERs who have been granted the DBA role"

Description: The DBA is typically defined as the UX group which has "connect internal" access. Any user which has access to this group therefore can perform "connect internal". So, the "oracle" or ora* userids are the only one which should belong to the DBA group.
Corrective Action: Un-enroll these users from the DBA group.

DBA316 - "The init.ora parm ? is (?) - It must be ? for security reasons"

Description: It has been determined that this init.ora parameter, set the way that you have chosen to set it, poses a security risk.
Corrective Action: Change the setting.

DBA317 - "Tablespace(s) Found With NO Datafiles: $F3"

Description: The tablespaces listed have no datafiles (or tempfiles).
Corrective Action: This should not happen under normal circumstances. Add a datafile or recover the missing datafile.

DBA318 - "There are 1 Non-System USERs who have been granted the ALTER USER priv"

Description: The named privilege was granted to a non-SYSTEM user.
Corrective Action: Revoke this privilege as appropriate.

DBA319 - "There are $dbarole_cnt Non-System USERs who have been granted the IMP_FULL_DATABASE role"

Description: The named privilege was granted to a non-SYSTEM user.
Corrective Action: Revoke this privilege as appropriate.

DBA320 - "DST2007 ($sw_version[$thishost]) JVM Is installed"

Description: This event was coded for the 2007 DST Change. This instance DOES have the JVM installed.
Corrective Action: This instance must be patched.

DBA321 - "DST2007 ($sw_version[$thishost]) SYS TZ Columns Founds"

Description: This event was coded for the 2007 DST Change. This instance DOES have the SYS-owned *TIME ZONE* columns.
Corrective Action: This instance must be patched.

DBA322 - "DST2007 ($sw_version[$thishost]) Non-SYS TZ Columns Found"

Description: This event was coded for the 2007 DST Change. This instance DOES have the Non-SYS-owned *TIME ZONE* columns.
Corrective Action: This instance must be patched.

DBA323 - "DST2007 ($sw_version[$thishost]) TZ-Column/JVM Patch Required and Missing"

Description: This event was coded for the 2007 DST Change. It was determined that for either "TZ-Columns" or JVM that the DST2007 patching is required, but not installed.
Corrective Action: This instance must be patched.

DBA330 - "CLEANERS ($cleaners[$thishost]) must be at least 75% of the number of disks ($num_disks[$thishost]) and >= (LRUS/2) (LRUS=$lrus[$thishost])"

Description: For good performance, the number of CLEANERS should be >= 75% of the number of disks that contain chunks. Also, the number of CLEANERS should be >= LRU's/2.
Corrective Action: Set the CLEANERS value to the correct setting and bounce Informix.

DBA330 - "HDR Not Active - Type=$hdr_type State=$hdr_state Name=hdr_name"

Description: DBAmon looked at the 'sysdri' SMI table and found that HDR was configured but not active.
Corrective Action: Restart HDR.

DBA340 - "Duplicate ONCONFIG Parameters Found"

Description: The ONCONFIG parms listed were specified twice. This probably means that you meant to change something, but did so in the wrong place.
Corrective Action: Remove the duplicate parameter(s).

DBA350 - "Current Average Checkpoint Duration is $ckptavg[$thishost] Seconds"

Description: The most recent average checkpoint duration was > 120 seconds.
Corrective Action: Probably due to too much data to write at checkpoint time, or too few cleaners. Try increasing CLEANERS and decreasing LRU_MIN_DIRTY and LRU_MAX_DIRTY to reduce the number of dirty pages when the checkpoint time arrives.

DBA360 - "(KAIO on) - NUMAIOVPS value of $numaiovps[$thishost] is greater than recommended value of 1 or 2"

Description: If KAIO is on, then you only need to specify 1 or 2 NUMAIOVPS to do file I/O.
Corrective Action: Reduce the number of NUMAIOVPS to 2.

DBA370 - "(KAIO off) - AIO Least to Most Ratio $aiorat[$thishost] > Threshold of 40 - NUMAIOVPS value: $numaiovps[$thishost] too small"

Description: In an attempt to help you tune the correct number of AIO VP's, DBAmon runs onstat -g iov to see how many I/O's have been issued by the most and least busy AIO VP. If the most busy is >= 40 times busier than the least busy, then more AIO VP's should be configured.
Corrective Action: The message contains a recommended number NUMAIOVPs. Change the ONCONFIG file to this value and bounce Informix.

DBA380 - "Normal Backup Schedule Not In Place"

Description: This Informix HDR Primary has no registered backups.
Corrective Action: Run a LVL0 backup.

DBA390 - "(Backup Running Now) $rounded1-$rounded2 Hours since last good onbar/ontape backup (threshold=$backup_age[$thishost])"

Description: You have specified the Backup_Age: parameter to turn on backup age checking. The age of the most recent ontape-style backup exceeds the threshold that you specified. However, a backup is running now, so this event is a WARNING.
Corrective Action: This event will go away when this backup ends successfully.

DBA390 - "(Backup Was Rerun) $rounded1-$rounded2 Hours since last good onbar/ontape backup (threshold=$backup_age[$thishost])"

Description: You have specified the Backup_Age: parameter to turn on backup age checking. The age of the most recent onbar/ontape-style backup exceeds the threshold that you specified. However, since you also specified Backup_Command: DBAmon has automatically launched a backup according to the command that you specified.
Corrective Action: This event will go away when this backup ends successfully.

DBA391 - "(Backup Running Now) $rounded1-$rounded2 Hours since last good onarchive backup (threshold=$backup_age[$thishost])"

Description: You have specified the Backup_Age: parameter to turn on backup age checking. The age of the most recent onarchive-style backup exceeds the threshold that you specified. However, a backup is running now, so this event is a WARNING.
Corrective Action: This event will go away when this backup ends successfully.

DBA391 - "(Backup Was Rerun) $rounded1-$rounded2 Hours since last good onarchive backup (threshold=$backup_age[$thishost])"

Description: You have specified the Backup_Age: parameter to turn on backup age checking. The age of the most recent onarchive-style backup exceeds the threshold that you specified. However, since you also specified Backup_Command: DBAmon has automatically launched a backup according to the command that you specified.
Corrective Action: This event will go away when this backup ends successfully.

DBA399 - This Oracle instances uses SPFILE - spfile=$spfile[$thishost]"

Description: This instance has a non-null value for the SPFILE init.ora parameter. This event only occurs if the Run_SPFile_Check: dbamonrc parameter is set to Y.
Corrective Action: This event will go away when this backup ends successfully.

DBA400 - " $mrl0days[$thishost] Days since last good Level=0 backup (threshold=$l0_age[$thishost])"

DBA410 - "Possible Long TX Detected - HWMPCT: $hwmpct (threshold=$t_longtx[$thishost])"

Description: See Long Transaction Detection.
Corrective Action:

DBA460 - "$msg"

Description: This event means that the number of UX file descriptors used is >= 90% of the kernel configured value. This will kill Informix if it reaches 100%.
Corrective Action: Stop processes that are using file descriptors, or increase the appropriate kernel parm.

DBA461 - "UX NFILEs %s%% Used - Current Value=%s Kernel Maximum=%s (Critical Threshold=95%%)"

Description: This event means that the number of UX file descriptors used is either >= 90% of the kernel configured value (Warning event) or >= 95% (Critical event). This will cause Oracle to crash if it reaches 100%.
Corrective Action: Stop processes that are using file descriptors, or increase the appropriate kernel parm.

DBA470 - "The Oracle Autostart software (/sbin/init.d/oracle) is obsolete - it does not invoke oraadmin"

Description: The current /sbin/init.d/oracle (as of 06/2005) invokes oraadmin. This file that was found does not invoke oraadmin.
Corrective Action: Run /usr/local/dba/tools/oracle_autostart/setup as root.

DBA471 - "This server does not have Oracle Autostart configured - Start and/or Stop symlink missing"

Description: The HP-UX autostart files (/sbin/rc2.d/K800oracle and /sbin/rc3.d/S200oracle) don't exist. So autostart is not configured.
Corrective Action: Run /usr/local/dba/tools/oracle_autostart/setup as root.

DBA472 - "The Oracle Listener Log $F5 is $F3 MB (exceeds threshold of $F4 MB)"

Description: The $ORACLE_HOME/network/log/listener.log file is TOO BIG.
Corrective Action: Remove or compress file.

DBA502 - "Undiagnosed rcp Error - Userid=$userid[$thishost]"

Description: DBAmon was attempting to download the DBAmon "Probe" software to this host, but the rcp command failed.
Corrective Action: Read the accompanying messages and solve the problem accordingly.

DBA502 - "rcp Error - Userid=$userid[$thishost] account disabled"

Description: DBAmon was attempting to download the DBAmon "Probe" software to this host, but the rcp command failed. The problem is that the account that we are rcp'ing to is disabled.
Corrective Action: Re-enable the account and retry.

DBA502 - "rcp Error - Userid=$userid[$thishost] login incorrect"

Description: DBAmon was attempting to download the DBAmon "Probe" software to this host, but the rcp command failed. The problem is that the .rhosts file on the remote for this userid does not have a correct entry for the DBAmon master.
Corrective Action: Add the appropriate .rhosts entry to the userid's account on the remote host.

DBA502 - "rcp Error - Userid=$userid[$thishost] password expired"

Description: DBAmon was attempting to download the DBAmon "Probe" software to this host, but the rcp command failed. The problem is that the password for this userid has expired.
Corrective Action: Set a new password.

DBA502 - "rcp error - userid=$userid[$thishost] Undiagnosable"

Description: DBAmon was attempting to download the DBAmon "Probe" software to this host, but the rcp command failed.
Corrective Action: Correct the remsh connectivity problem.

DBA503 - "$perlpath[$thishost] Does Not Invoke Perl Version 5"

Description: DBAmon checks the Perl version on the remote host to ensure that it is >= version 5.
Corrective Action: Upgrade Perl.

DBA503 - "/usr/local/bin/perl Does Not Invoke Perl Version 5"

Description: DBAmon checks the Perl version on the remote host to ensure that it is >= version 5.
Corrective Action: Upgrade Perl.

DBA504 - "Unable to find ORACLE_SID=$orasid[$thishost] in /etc/oratab - Unable to convert * to value"

Description: You specified ORACLE_HOME: of * in the DBC file. While DBAmon was attempting to resolve this to a specific ORACLE_HOME value, it looked in /etc/oratab for an entry with the SID that specified. This entry did not exist.
Corrective Action: Create the correct entry in /etc/oratab on the DB server.

DBA505 - "Filecheck: File=$filecheck Not Found"

Description: You must have specified the Filecheck: dbamonrc parm. DBAmon did not find the file that you specified.
Corrective Action: Place the file on the DB server.

DBA511 - "$rounded1-$rounded2 Hours since last good ora_backup backup - threshold: $backup_age[$thishost] $running_str"

Description: You specified the Backup_Age: DBC parameter for this instance. The number of hours since the last 'ora_backup' backup exceeds the threshold that you specified.
Corrective Action: Run a backup.

DBA512 - "$bckmsg - threshold: $backup_age[$thishost] $running_str"

Description: Similar to DBA511.
Corrective Action: Run a backup of the correct type.

DBA513 - "Backup Method for this DB is $backup_method_long[$thishost] - Backup_Age: requires that Backup Method be RMAN; EXP; TBS or FULL"

Description: You specified the Backup_Age: DBC parameter for this instance. DBAmon then tried to determine the DB backup type for this instance. Is must be one of the types listed above.
Corrective Action: Run a backup of the correct type.

DBA514 - "MSSQL DB=$db - Backup Was Invoked - Method=$amethod"

Description: You specified the Backup_Age: DBC parameter for this MSSQL instance. The number of hours since the last backup exceeds the threshold that you specified.
Corrective Action: Run a backup.

DBA515 - "MSSQL DB=$db - Invoked Backup Failed - Method=$amethod"

Description: You specified the Backup_Command: DBC parameter. DBAmon tried to invoke a backup, but it failed.
Corrective Action: Examine the accompanying error messages and act accordingly.

DBA516 - "Backup Method for this DB is $backup_method_long[$thishost]"

Description: The backup type for this Oracle DB is 'NONE'. If you specify Backup_Age: in the DBC file, then you should have backups scheduled.
Corrective Action: Schedule backups.

DBA517 - "$bckmsg - Threshold: $backup_age_lvl0[$thishost] (Hours) $t_lvl0_days (Days) $running_str_lvl0"

Description: The number of hours since the last successful RMAN LVL0 backup exceeds the threshold. The threshold is calculated by multiplying the Backup_Age: DBC parameter (specified in hours) by 7. If the resulting number is > 15 days it is set to 15 days.
Corrective Action: Run an RMAN LVL0 backup. To prevent this event, set the Backup_Command: DBC parameters to run a LVL0 backup. Then DBAmon will automatically run a LVL0 backup when this threshold is exceeded.

DBA518 - "There have been $unrecdf_cnt UNRECOVERABLE Datafile Changes since the last RMAN LVLx backup"

Description: This event occurs when an UNRECOVERABLE (NOLOGGING) change has been made to the database since the last LVL0 backup. You will not be able to roll the entire database forward past the time of the unrecoverable change without corrupting data if you have to RECOVER the database.
Corrective Action: Run an RMAN LVL0 backup and stop making unrecoverable changes to the database.

DBA520 - "Redo Log Switch Rate for the last 24 hours is $redo_1day_count[$thishost] Switches/Hour (Threshold: $df_redo_rate) - Excessive log switches are BAD for DB performance - Increase Online Redo Log size"

Description: DBAmon measures the redo log switch rate for the last 24 hours. That number is compared to the the REDO SWITCHES PER HOUR threshold. If that threshold is exceeded, then this event occurs. Excessive log switches are BAD for DB performance.
Corrective Action: To reduce the number of Online Redo Log switches, increase the size of the Online Redo Logs.

DBA521 - "Peak Last-30-Days Redo Rate ($redo_30day_max_gb[$thishost] GB) vs. Archivelog FS Size ($redo_archv_fs_gb[$thishost] GB)Ratio is $redo_archv_ratio[$thishost] (Threshold=$df_redo_fs_ratio) - Increase Archivelog FS Size to ${z} GB "

Description: It is a good pratice to size the archivelog FS of an Oracle instance to hold 1 PEAK Days worth of redo data. It was determined that the archivelog FS of this instance does not meeting this criteria (it is too small)
Corrective Action: Increase the size of the archivelog filesystem.

DBA601 - "Oracle Max Processes Exceeded: $reason"

Description: SQL could not run because processes has been exceeded.
Corrective Action: Solve the cause of the excessive processes.

DBA602 - "Oracle Not Active/DB Not Open: $reason"

Description: Oracle is down.
Corrective Action: Restart Oracle.

DBA603 - "Oracle Crashed - $z"

Description: DBAmon found from the end of the alert log that Oracle has crashed. Depending on the error and the setting of the DBC parameter Must_Be_Up: DBAmon may have tried to restart Oracle.
Corrective Action: Restart Oracle if DBAmon did not already do this for you.

DBA604 - "Online Redo Log $F2 $F3 In Exception Status"

Description: The online redo log mentioned is not in normal status.
Corrective Action: Examine the alert log for the cause of the problem.

DBA605 - "Oracle PROCESSES - Current Count: $current - INIT.ORA Value: $config - Percent Used: ${procpct}%"

Description: The current number of processes (from v$process) is this percent of the init.ora "processes" parameter.
Corrective Action: Get rid of some sessions before you use them all up.

DBA606 - "Oracle DB_FILES - Current Count: $f_current - INIT.ORA Value: $f_config - Percent Used: ${f_pct}%"

Description: The current number of datafiles (from v$datafile) is this percent of the DB_FILES init.ora parameter.
Corrective Action: Drop unneeded tablespaces or increase DB_FILES (this requires an instance bounce).

DBA607 - "Tool ora_oddjob not found in crontab"

Description: The ora_oddjob tool was not found in cron (crontab -l was run).
Corrective Action: Add a correct entry to cron for ora_oddjob.

DBA608 - "Tool ora_backup_sched not found in crontab"

Description: The ora_backup_sched tool was not found in cron (crontab -l was run).
Corrective Action: Add a correct entry to cron for ora_backup_sched.

DBA610 - "Critical Messages Found in Oracle Alert Log"

Description: DBAmon looks for certain strings in the last 20 lines of the alert log. It found at least of these strings there. The strings that DBAmon looks for can be found in /opt/dbamon/bin/download/dbamon_ora.cfg
Corrective Action: Solve the Oracle problem.

DBA611 - "Listener Not Active: $lsnrerror"

Description: When DBAmon checked to see if the listener was running, it issued: lsnrctl status. This is what failed.
Corrective Action: Get the listener running for this DB.

DBA612 - "Tnsping failed: $tnserror"

Description: When DBAmon checked to see if the listener was running, it issued: tnsping $ORACLE_SID.world. This is what failed. The accompanying error message should give a hint as to what the problem is.
Corrective Action: Get tnsping $ORACLE_SID.world to work for this DB.

DBA613 - "Listener Was Restarted"

Description: DBAmon found that the default listener was not runing. It attempted to issue 'lsnrctl start' and it worked.
Corrective Action: This is an informational message.

DBA614 - "Listener Restart Failed"

Description: DBAmon found that the default listener was not runing. It attempted to issue 'lsnrctl start' and it failed.
Corrective Action: Examine the accompanying error messages and act accordingly.

DBA615 - "Permissions changed on listener.ora to 700"

Description: DBAmon changed the file permissions of listener.ora to 700.
Corrective Action: Informational Message.

DBA616 - "Password established on listener.ora"

Description: DBAmon established a non-encrypted password in the listener.ora file.
Corrective Action: Informational Message.

DBA621 - "Oracle TS: $ts - ${pct}% Full - Critical ($mbfree MB Free - Threshold: ${t_diskcrit[$thishost]}%)"

Description: The tablespace mentioned is full or almost full.
Corrective Action: Add a datafile.

DBA622 - "Oracle TS: $ts - ${pct}% Full - Warning ($free MB Free - Threshold: ${t_diskcrit[$thishost]}%)"

Description: The tablespace mentioned is full or almost full.
Corrective Action: Add a datafile.

DBA623 - "Add Datafile Command Failed - rc=$F2 cmd=$ts_command[$thishost] ts=$ts pc=$pc ad=$ad"

Description: You have specified the T_TS_Command: in the DBC file for this DB. At least one tablespace was at least the Warning Threshold full, so DBAmon invoked this command that you specified. This event means that this command ended with a non-zero return code.
Corrective Action: Solve the problem which is preventing your Add Datafile command from working.

DBA624 - "Added Datafile to TS: $ts - Was: ${pc}% Full - Added: $ad MB"

Description: You have specified the T_TS_Command: in the DBC file for this DB. At least one tablespace was at least the Warning Threshold full, so DBAmon invoked this command that you specified. This event means that this command ended with a zero return code.
Corrective Action: None. This event is notification that your Add Datafile command worked.

DBA625 - "Oracle Error encountered while checking tablespaces"

Description: DBAmon encountered a critical error while trying to check your tablespaces. Look at the accompanying diagnostic messages.
Corrective Action: Solve the problem which is preventing this from working.

DBA626 - "Tablespace $F2 Was Coalesced"

Description: This tablespace was full or almost full - DBAmon automatically coalesced it.
Corrective Action: None. If this did not free space, then you will have to add space.

DBA627 - "RBS Segments In Tablespace $F2 Were Shrunk "

Description: The RBS tablespace was full or almost full. DBAmon automatically shrank the RBS segments that reside there.
Corrective Action: None. If you don't want this to happen, don't let RBS fill!

DBA630 - "Object(s) Found With Extents >= ${t_extents[$thishost]}% Of Max_Extents"

Description: You specifed the "T_Extents:" DBC parameter of X. The tables in question have >= X percent of maxextents.
Corrective Action: You can run reorg the table to 1 extent, or ALTER TABLE x MAXEXTENTS UNLIMITED or let DBAmon do this for you by specifying:
T_Extents: x Fix

DBA631 - "Oracle In RESTRICTED SESSION Mode:"

Description: Oracle is in RESTRICTED SESSION mode.
Corrective Action: Run ALTER SYSTEM DISABLE RESTICTED SESSION.

DBA632 - "Found $foundtbls Whose Next_Extent Will Not Fit"

Description: The objects listed have a next extent size that will not fit in the indicated tablespace. The issue is that there is not an area of CONTIGUOUS freespace large enough in the tablespace. This can also be caused by PCTINCREASE being set to non-zero. It has been my experience that this is a bad practice to ever set PCTINCREASE to a non-zero value. It causes runaway extent size and non-uniform "holes" in tablespaces.
Corrective Action: One of:

Reduce the size of the next extent:
```
ALTER TABLE OWNER.TABLE STORAGE ( NEXT ?M ); 
```
... so that the next extent size is less than the largest freespace area.
-Or
Add an amount of space to the tablespace that is greater than the size of the next extent.

Also, if PCTINCREASE is non-zero, set it to 0. ALTER the object in question so that the next extent size is less than the largest contiguous piece of freespace. Then, contact the business partner to inform them about what you have done. Advise the BP that if they have an issue with our taking this action that they need to open a ticket to us to discuss alternatives. Also, if the object has PCTINCREASE set to non-zero, inform them that you intend to set it to 0. The reason for asking the BP is that they may have intentionally specifed a large NEXT EXTENT size.

DBA640 - "Oracle ORACLE_HOME FS Is Full/Almost Full ({$homefull[$thishost]}%)"

Description: The disk/filesystem where ORACLE_HOME reside is full or almost full.
Corrective Action: Free some disk space.

DBA641 - "Oracle Archive Log Dir: $arcdir FS is ${arcdirfull}% Full - Warning (Threshold: ${t_arclog_w[$thishost]}%)"

Description: The disk/filesystem containing the archive log destination is almost full.
Corrective Action: Make some space on the disk before it fills!

DBA642 - "Oracle Archive Log Dir: $arcdir FS is ${arcdirfull}% Full - Critical (Threshold: ${t_arclog_c[$thishost]}%)"

Description: The filesystem containing the archive log destination is almost full.
Corrective Action: Make some space on the disk before it fills!

DBA643 - "Oracle Archive Logging is On; But Auto Archiving is Off"

Description: Having archivelog mode on and auto archiving off doesn't make sense.
Corrective Action: Either turn on auto archiving (set log_archive_start = true in init.ora) or turn off archivelog mode.

DBA644 - "Either: (1) NT srvinfo Command Did Not Run -or- (2) NT srvinfo Command Did Not Find The ARCLOG Disk -- Something Is Wrong"

Description: DBAmon ran the Toolkit srvinfo command to see how full the disk where the archive log reside is. It failed.
Corrective Action: If srvinfo is not found, install the Microsoft NT Resource Kit.

DBA645 - "Oracle Archive Log Dir is NULL: Something went wrong with svrmgrl"

Description: While DBAmon was checking value of log_archive_dest, a null value was returned. This could be because Oracle is DOWN.
Corrective Action: The next time that DBAmon checks to see if Oracle is up, this condition will be further diagnosed.

DBA646 - "Oracle Mandatory Archive Log Destination(s): $arcdestlist in ERROR Status"

Description: An Archive Destination with a binding of MANDATORY was found to be in ERROR status.
Corrective Action: Correct the cause of the error and issue the appropriate ALTER SYSTEM command to cause Oracle to reopen this destination.

DBA647 - "Oracle Archive Log OPTIONAL Destination(s) (with reopen > 0): $arcdestlist in ERROR Status"

Description: An Archive Destination with a binding of OPTIONAL and REOPEN > 0 was found to be in ERROR status.
Corrective Action: Correct the cause of the error and wait for Oracle to automatically reopen this destination.

DBA648 - "OFFLINE Datafile(s) Found"

Description: For some reason, you have datafiles that are not ONLINE.
Corrective Action: Examine the alert log and act accordingly.

DBA641 - "Oracle Archive Log Dir: $arcdir FS $arcfs is ${arcdirfull}% Full - Warning (Threshold: ${t_arclog_w[$thishost]}%) $rerun"

Description: The archivelog FS named here is above the warning threshold full.
Corrective Action: Run the appropriate process to reduce the amount of space used in this filesystem.

DBA649 - "init.ora Error: $z"

Description: A contradiction was found in your init.ora file.
Corrective Action: Fix it!

DBA650 - "Archivelog $F3 has invalid format - Should be arch%t_%s.dbf"

Description: At least 1 archivelog was found in one of your archivelog destinations whose format was either:

filesystemname/1_NNNNN.dbf
filesystemname/archarchv1_NNNNN.dbf

... which violates our standard of /arch%t_%s.dbf.
Corrective Action: Change init.ora log_archive_dest* and/or log_archive_format. This can be done dynamically with alter system in 8i+.

DBA651 - "MSSQL Eventlog Alert(s) Found"

Description: While DBAmon was examining the MSSQL Alert Log, it found this message with a severity of 17 or higher.
Corrective Action: Act according to the error message text.

DBA652 - "$dbmsout[$thishost] Version $sw_version[$thishost] is older than the minumum 'good' version ($this_minver) for this family ($this_family)"

Description: This message originates with DBAmon DBMS Version Oversight. It only appears if you have configured it from the DBAmon Console. This message means that the indicated DBMS instance is running a version of the vendor DBMS software which is lower than the version that you have specified as the "Minimum Good" versoin for this version family. For example, if you have configured Oracle 8.1.7.4 as the "Minimum Good" version for the 8.1.7 family, any instance that is running 8.1.7, but less than 8.1.7.4 will receive this event.
Corrective Action: Upgrade the instance to a higher software version.

DBA653 - "The filesystem(s) that match $fs_check_mask[$thishost] are $pct_fsused[$thishost]% full (threshold : $fs_check_threshold"

Description: This event is created the FS Full Checking. The filesystems which match the FS_Check_Mask: are >= the FS_Check_Threshold: .
Corrective Action: Remove files from the filesystems or add space.

DBA654 - "All [controlfiles/redo logs] are one 1 Drive - They should be spread out to multiple drives"

Description: It is a poor practice to place all controlfiles/redo logs on 1 disk.
Corrective Action: Make sure that the controlfiles/redo logs are placed on different disks.

DBA655 - "All redo logs are one 1 Drive - They should be spread out to multiple drives"

Description: It is a poor practice to place all controlfiles/redo logs on 1 disk.
Corrective Action: Make sure that the controlfiles/redo logs are placed on different disks.

DBA656 - "DB=$tl_dbname - TLog is $tl_ratio times DB size and TLog is >= 1gB (DBSize=$tl_dbsize (mB) TLSize=$tl_tlsize (mB) Threshold=$tl_ratio_threshold)"

Description: The transaction log has grown to at least 1gB in size and is at least 2 times the size of the database datafiles. As this does not make sense, it indicates a problem where, probably, the transaction log is not getting backed up and cleaned out.
Corrective Action: You need to recreate the transaction log AND ensure that backups start running on a regular basis so as to prevent this from reoccuring.

DBA657 - "DB=$tl_dbname - TLog with LIMITED size ${this_tl_full_vs_limit}% full ($event_sev_long Threshold of $event_pct% exceeded - TLSize=${tl_tlsize}(mB) TLLimit=${tl_growth}(mB))"

Description: The transaction log is full or almost full. This particular TLOG has a size limit. So, it is >= 90% full internally, and it has reached or almost reached its limit.
Corrective Action: Backup the transaction log.

DBA658 - "DB=$tl_dbname - TLog with UNLIMITED growth - Drive $tl_drive is $tl_drivepct full ($event_sev_long Threshold of $event_pct% exceeded - TLSize=$tl_tlsize(mB) TLPath=$tl_filename)"

Description: The transaction log is full or almost full. This particular TLOG does not have a size limit. So, it is >= 90% full internally, and the disk where the TLOG resides is full or almost full.
Corrective Action: Backup the transaction log.

DBA659 - "You only have $cf_rows - You should have at least $df_cf_min (cf_rows=$cf_rows)"

Description: In Oracle it is a good practice to have >1 controlfile (there's no good reason not to). So DBAmon will monitor the number of controlfiles for you. If you have the Default_Min_CF: dbamonrc parameter set, then the actual number of controlfiles will be compared against the value that you specify. If the number of controlfiles is less, then this event will occur.
Corrective Action: Add controlfile(s). To do this, stop the instance, copy the existing controlfile to the new controlfile filenames, change init.ora to include your new controlfile in the "control_files" parameter, and start the instance.

DBA660 - "Oracle SGA is $sga_pct[$thishost]% Full ($sev threshold: $t_sga_c[$thishost]%)"

Description: The Oracle SGA is full or nearly full.
Corrective Action:

Workaround: Run "alter system flush shared_pool". This will clear all data from the shared pool. However, if the activity on the DB is similar afterwords to the activity that filled the shared pool, then the condition will likely return. Flushing the shared pool does de-fragment it.
Long Term Fix: Increase the "shared_pool_size" init.ora parameter.

DBA661 - "User=$userid[$thishost] crontab is empty"

Description: This crontab is empty.
Corrective Action: Populate cron.

DBA662 - "Archivelog destination(s) ($F3) are in ORACLE_HOME filesystem"

Description: The default archivelog destination is $ORACLE_HOME/dbs. This is not a good practice to put archivelogs into the $ORACLE_HOME filesystem.
Corrective Action: Set the appropriate archive_log_dest_N init.ora parameter. In the case of a standby database, you will need to set standby_archive_dest.

DBA663 - "Non-System DB Count is Zero"

Description: There aren't any Non-System (msdb, master, model, tempdb) databases. Why have an instance if you don't have any data in it?
Corrective Action: Create a Database.

DBA665 - "This Standby DB is $delta Minutes Behind Primary (Threshold is $is_threshold[$thishost])"

Description: You have invoked Standby In-Sync checking by specifying the In_Sync* parameters in the DBC file for this instance. DBAmon compares the CONTROLFILE_TIME in v$database values of the standby database to the same value of the primary database. If these two values differ more than the In_Sync_Age: value that you specfied (in minutes) then this event will occur. In other words, The amount of time since the standby DB has been refreshed exceeds the In_Sync_Age: parameter that you specified.
Corrective Action: Your process for updating the standby DB is not working. Fix it.

DBA666 - "$nologobj NOLOGGING Object(s) Found On Primary DB $is_prihost[$thishost]/$is_prisid[$thishost]"

Description: You have invoked Standby In-Sync checking by specifying the In_Sync* parameters in the DBC file for this instance. In order for a standby database to be kept in sync with its primary, there cannot be any tables or indexes in the primary DB which were created with the NOLOGGING parameter. This event means that DBAmon has detected the existence of at least on NOLOGGING table or index on the primary database.
Corrective Action: Just because there are NOLOGGING objects on the primary DB, this does not necessarily mean that there has been a NOLOGGING operation on the primary DB which would invalidate the standby DB. You must alter the NOLOGGING object(s) to LOGGING. A DBA668 event will occur if an UNRECOVERABLE change was made to the primary.

DBA667 - "Primary DB $is_prihost[$thishost]/$is_prisid[$thishost] is DOWN (SQL will not run)"

Description: You have invoked Standby In-Sync checking by specifying the In_Sync* parameters in the DBC file for this instance. In order for a standby database to be kept in sync with its primary, the primary DB, whose server and ORACLE_SID you have specified in the DBC file, is down.
Corrective Action: Start the primary DB, or correct the DBC parameters that specify the primary DB.

DBA668 - "Primary DB Has $priunrec datafile(s) with UNRECOVERABLE changes since the last rebuild"

Description: You have invoked Standby In-Sync checking by specifying the In_Sync* parameters in the DBC file for this instance. DBAmon has detected that an unrecoverable change has occured on the primary which therefore was not transmitted to the standby. This unrecoverable change has occured after the most recent rebuild of the standby database.
Corrective Action: Rebuild the standby DB.

DBA680 - "Tablespace $zts Datafile Count $zdfc Approaching Maxiumum of 1022 (${zpct}%) - Thresholds W/C $t_dfcount_pct_w[$thishost]/$t_dfcount_pct_c[$thishost] (?)"

Description: There is a tablespace whose datafile count is approaching the 1022 datafile per tablespace Oracle limitation (for non-bigfile tablespaces).
Corrective Action: DO NOT allow this tablespace to hit 1022 datafiles.

DBA690 - "LAN Interface $lancard is in Half-Duplex mode ($lanmsg)"

Description: By running 'lanadmin -x ?' DBAmon found that this lan interface is running 100mb Half-Duplex.
Corrective Action: The UX sysadmin needs to reconfigure this lan interface to run full-duplex.

DBA701 - "Program Error: $F0 - $msg"

Description: While trying to check the status of your OracleApps instances, an error was encountered.
Corrective Action: Depends on text of message.

DBA701 - "OracleApps $proctype Process(es) Missing - Found: $proccnt MinThreshold: $procthr"

Description: The number of OracleApps processes of the specified type was less that the threshold minimum number of processes that you specified in our DBC file.
Corrective Action: Restart missing processes or reduce minimum process threshold in DBC file.

DBA702 - "Critical OracleApps Processes: $errorprocesses Not Active"

Description: Process(es) of the specified type should be running, but they are not.
Corrective Action: Restart the process.

DBA740 - "EM/iSQLPlus emctl Was restarted OK"

Description: DBAmon automatically restarted EM or iSQLPlus.

DBA801 - "MSSQL Not Active: $dbstatus $dbmsg"

Description: MSSQL was found to be down.
Corrective Action: Restart MSSQL.

DBA802 - "$oldperl_cnt Old Perl process(es) found - Max process age: $oldperl_maxage hours - Threshold: ? hours"

Description: DBAmon looks for Perl.exe processes that have been started by the same userid that the DBAmon probe runs under that have been running for at least 24 hours. Any such processes will be listed in the long text of this event. It is assumed that any Perl process that has been running for at least 24 hours is hung and will not finish without "help"; If you do have any Perl long running jobs, such as services, run them under a different userid that the one that the remsh service runs under.
Corrective Action: Run the "pskill" commands that appear with the long text to kill these processes.

DBA803 - "MSSQL Agent Service Not Active"

Description: The MSSQL Agent process in not active.
Corrective Action: Start the MSSQL Agent.

DBA804 - "MSSQL Active But DB $F2 Is OFFLINE"

Description: A DB is offline.
Corrective Action: Bring the specified DB online.

DBA805 - "Drive $dr_drive is $dr_pct full ($sev threshold: $t_disk$sev[$thishost]%)"

Description: The Drive mentioned contains at least one MSSQL database file and is full or almost full.
Corrective Action: Add space to this drive or remove unneeded files.

DBA810 - "MSSQL DB=$db - Backup Never Run - Threshold=$backup_age[$thishost] $bckrerun_msg"

Description: A backup was never run for the DB listed.
Corrective Action: Run a backup for this DB.

DBA811 - "MSSQL DB=$db - $rounded1-$rounded2 Hours Since Last Good Backup - Threshold=$backup_age[$thishost] $bckrerun_msg"

Description: The number of hours since the most recent successful backup exceeds what you specified in the "Backup_Age:" DBC parameter.
Corrective Action: Run a backup.

DBA811 - "$rounded1-$rounded2 Hours Since Last Good Backup (Threshold=$backup_age[$thishost])"

Description: The number of hours since the most recent successful backup exceeds what you specified in the "Backup_Age:" DBC parameter.
Corrective Action: Run a backup.

DBA901 - "Oracle/NT Not Active - Status=$dbstatus"

Description: Oracle is down.
Corrective Action: Start Oracle.

DBA903 - "Oracle/NT Listener Not Active - Status=$lsnr_status"

Description: DBAmon attempted to run 'tnsping $ORACLE_SID'. It failed.
Corrective Action: Determine the cause of the problem and fix it!

DBA905 - "Connect Logic Error - Probe dbamon_orant.pl Did Not Finish"

Description: A DBAmon probe module failed.
Corrective Action: Examine accompanying messages. Contact DBAmon Support.

DBA909 - "DBAMON.TIMESTAMP Has > $timestamp_rowlimit rows ($timestamp_rows[$thishost])"

Description: The number of rows in this table exceeds the threshold. The purge process must not be working.
Corrective Action: Contact BB.

DBA910 - "db_block_buffer Read Hit Ratio of $bufhitratio[$thishost] < threshold of $t_readhit[$thishost]"

Description: The Oracle db_block_buffer hit ratio (specifed in message) is below the threshold specified for this instance by the T_Read_Hit: DBC parameter (or the default). This DB is performing poorly.
Corrective Action: Increase the db_block_buffers init.ora parameter.

DBA911 - "db_block_buffer Read Hit Ratio of $bufhitratio[$thishost] is invalid (< 0 or > 100)"

Description: While examining the Oracle db_block_buffer hit ratio, the value was found to be invalid. Check the accompanying text to see if some error occured while querying the Oracle dictionary.
Corrective Action: Solve the problem which was causing the query to return invalid data.

DBA912 - RMANHUNG: Found rman process (pid=$) that has been running for $ days (threshold=2 days) !!!

Description: A Unix process containing the string rman was found to have been running for more than the THRESHOLD-DAYS number of days long. It is probably a dead process which may be consuming resources even though it is not doing anything useful.
Corrective Action: Kill the hung process at the UX level. If RMAN was invoked from a backup script, also make sure that you kill the script.

DBA913 - $num_otrace[$thishost] ORACLE_HOME/otrace/admin/*.dat Files Found - OTRACE Is ON Which Causes Performance Problems!!!

Description: OTRACE is on for this DB because .dat files were found in ORACLE_HOME/otrace/admin. OTRACE can be bad for performance, so it should be turned off. You turn it off by rm'ing the .dat files in ORACLE_HOME/otrace/admin and restarting Oracle.
Corrective Action: Turn off OTRACE. You turn it off by rm'ing the .dat files in ORACLE_HOME/otrace/admin and restarting Oracle.

DBA914 - Instance SQL_TRACE=TRUE - This Causes Performance Problems!!!

Description: SQL_TRACE set to true at the instance level will cause serious performance problems.
Corrective Action: Turn off SQL_TRACE. You turn it off by running ALTER SYSTEM SET SQL_TRACE=FALSE And/Or removing this setting from init.ora.

DBA915 - $dfltsys_cnt DB Users Found With DEFAULT TABLESPACE Set To SYSTEM !!!

Description: DBAmon found DB users whose default tablespace is SYSTEM. This is very bad for performance.
Corrective Action: Alter the user so that their default tablespace is not SYSTEM.

DBA916 - $tempsys_cnt DB Users Found With TEMPORARY TABLESPACE Set To SYSTEM !!!

Description: DBAmon found DB users whose temporary tablespace is SYSTEM. This is very bad for performance.
Corrective Action: Alter the user so that their temporary tablespace is not SYSTEM.

DBA917 - $tempperm_cnt DB Users Found Whose TEMPORARY TABLESPACE Is a PERMANENT Tablespace !!!

Description: DBAmon found DB users whose temporary tablespace is permanent tablespace. This is very bad for performance.
Corrective Action: Alter the user so that their temporary tablespace is a TEMP tablespace, or alter their temp tablespace to be a type=TEMP tablespace.

DBA918 - * oraUp() DBA_REGISTRY Shows that component (Oracle9i Catalog Views ) is at version 9.2.0.2.0 which is less than DB engine version 9.2.0.3.0(64) - rg_status=VALID *

Description: Starting in 9i, the Oracle dictionary catalog contains components that are registered products. This event can also occur for DB internal components like Java. The event means that the version of the product mentioned is lower than the version of the DB engine. What probably happened is that the DB was upgraded within the same version (9i for example) and catpatch was not run.
Corrective Action: Run:

shutdown immediate
startup migrate
@?/rdbms/admin/catpatch
shutdown immediate
startup
Next, verify that the versions of the internal components match the DB engine version. Run orastat -rg.

DBA919 - MTS is being used for this Non-RAC/OPS instance - Bad for performance - mts_queue=$mts_queue[$thishost]

Description: This instance is not using RAC or OPS, but MTS is being used. This can cause major performance problems. So, it would be best to only use DEDICATED SERVER connections.
Corrective Action: An easy way to effectively disable MTS is to set USE_DEDICATED_SERVER=ON in sqlnet.ora.

DBA920 - TEMP Tablespace is a PERMANENT Tablespace (It Should Be TEMPORARY) !!!

Description: DBAmon found that your tablespace named TEMP is a permanent tablespace. This can be very bad for performance. Any user which has this tablespace specified for its TEMPORARY TABLESPACE will perform poorly when doing disk sorts.
Corrective Action: Alter the TEMP tablespace so that it is a type=TEMP tablespace. SQL:

ALTER TABLESPACE TEMP TEMPORARY;

DBA921 - Library Cache Hit Ratio is $libhitratio[$thishost]% (Should Be >= 90%) - Increase shared_pool_size !!!

Description: The Oracle library cache hit ratio was found to be < 90%.
Corrective Action: Increase the init.ora shared_pool_size parameter.

DBA922 - Dictionary Cache Hit Ratio is $dicthitratio[$thishost]% (Should Be >= 90%) - Increase shared_pool_size !!!

Description: The Oracle dictionary cache hit ratio was found to be < 90%.
Corrective Action: Increase the init.ora shared_pool_size parameter.

DBA923 - Rollback Segment Header Waits Are Too High (Gets/Waits Ratio > 1.00%) - Add Rollback Segments

Description: The total number of Rollback Segments Header Waits exceed 1% of the total number of Rollback Segment Header Gets.
Corrective Action: Add more rollback segments.

DBA924 - There are fewer than 10 Free DB Cache Buffers (free_buffers=$free_buffers[$thishost]) - It would be beneficial to increase DB Cache Size

Description: The total number of FREE DB Cache buffers is less than 10. So, this DB would probably benefit from you specifying a large DB buffer cache.
Corrective Action: If you have enough unused memory, increase the size of the DB Buffer Cache.

DBA925 - UNDO_MANAGEMENT Is not set to AUTO - It should be - undo_mgmt=$undo_mgmt[$thishost]

Description: In Oracle 9i and higher, SMU (System Managed UNDO) is a GOOD THING. It should always be on. For this 9i+ instance, it is not turned on.
Corrective Action: Turn on SMU. This will require DB downtime.

DBA926 - FORCE LOGGING Should be turned on - force_logging=$force_logging[$thishost]

Description: There is a very nice feature in Oracle 9 and higher called FORCE LOGGING. If this DB option is ON, then NOLOGGING operations are all automatically disallowed.
Corrective Action: Run: ALTER DATABASE FORCE LOGGING

DBA927 - ? Dictionary Objects have been ANALYZED - This is bad for performance

Description: Analyzing the Oracle Dictionary can cause some very serious and hard to diagnose performance problems. One symptom is high "recursive CPU" in a statspack report. If any SYS objects other than DUAL or PLAN_TABLE have been analyzed, this event will occur.
Corrective Action: Remove the analyze data for the SYS objects. Run: execute dbms_stats.delete_schema_stats('SYS');

DBA928 - DB Cache is only 1 granule in size - It was probably underspecified - granule_size=$granule_size[$thishost] db_cache _size=$db_cache_size[$thishost]?

Description: It was determined that this 9i or higher instance has a DB Buffer Cache that is only 1 granule in size. This can occur when Oracle sees that you have specifed a db_cache_size that is less than 1 granule. In this case Oracle will round up to 1 granule.
Corrective Action: It is better to intentionally specify the cache size. And the default is 48M, so a good minimum db_cache_size is 50M. Change the init.ora file and bounce the instance.

DBA929 - Server Memory is $hw_memory_pct_full[$thishost] used (t=$hw_memory_threshold_w/$hw_memory_threshold_c physmem=$h w_memory_size_gb[$thishost](gB) memfree=$hw_memory_free[$thishost](gB))

Description: This server has very little free memory. On HP-UX, this is bad for performance (paging and swapping increase).
Corrective Action: Reduce memory usage or increase the amount of memory. If you have any Oracle instances with overallocated SGA memory, reduce memory consumption.

DBA930 - The instance default PERM tablespace is set to SYSTEM - Could cause performance problems

Description: The DEFAULT TEMPORARY or PERMANENT tablespace is set to SYSTEM. If non-dictionary objects are create in SYSTEM, performance problem probably will result.
Corrective Action: ALTER DATABASE DEFAULT TABLESPACE tsname; (10g+ only)-or- ALTER DATABASE DEFAULT TEMPORARY TABLESPACE tsname; (9i+ only)

DBA931 - RMAN Process $F5 was automatically KILLED

Description: An RMAN process was found running on this server which:

Has a Parent Pid of 1 (this indicates that it has become orphaned)
Has been running for at least 5 minutes
Is consuming >= 50% of 1 CPU

This is indicative of an orphan RMAN process which is consuming CPU resources and is not accomplishing anything useful. DBAmon has issed the OS kill command against this process.
Corrective Action: (None)

DBA932 - UX NUSERPROC (HP-UX maxuprc Kernel Value) $os_nuserproc_pct[$thishost]% Used - MAXUPRC=$os_maxuprc[$thishost] OSUserProcCount=$os_ nuserproc[$thishost] (Thresholds: $t_nuserproc_w[$thishost]%/$t_nuserproc_c[$thishost]%)

Description: This event is unique to HP-UX. There is a UX kernel parameter maxuprc. This parameter controls the number of OS processes that any 1 UX userid can have running concurrently. If this is exceeded, the OS will not fork any new processes until the process count is reduced below this value. This can be VERY BAD for a running DB. DBAmon monitors the current OS process count against the configured maxuprc kernel value as a percentage.
Corrective Action:

Short Term: Kill any unneeded process that are owned by this user.
Long Term: Increase the maxuprc HP-UX kernel parameter.

DBA934 - Complex DB User Passwords Enforced

Description: Information only event. This DB has a UTLPWDMG routine active (password_verify_funtion) in the DEFAULT profile.
Corrective Action: No action required.

DBA940 - DB Block Corruption - $F3 Block segments

Description: Rows were found in V$DATABASE_BLOCK_CORRUPTION.
Corrective Action: Restore the corrupted blocks or datafiles or drop corrupt datafile.

DBA941 - DB Has $autoextend_cnt[$thishost] AUTOEXTEND=YES datafiles

Description: This DB has at least 1 datafile with Autoextend set to YES. This makes it impossible for DBAmon to monitor for full tablespaces.
Corrective Action: This event does not indicate a problem with your DB, but DBAmon will only monitor for tablespace full if you disable this attribute for all tablespaces.

DBA949 - COMPATIBLE Version ($this_compat) is < DBMS software version ($this_ver)

Description: The COMPATIBLE parameter is set a full version lower than the version of the DBMS software.
Corrective Action: Set COMPATIBLE to the same version as the DBMS software.

DBA955 - "IO Slave Count Of $numioslave Is >= $maxioslave_pct% of $maxioslave_cnt Maximum"

Description: The number of I/O slave processes is approaching 40. This is probably caused by hung RMAN processes, or dbwr_io_slaves set near 40 (the maximum).
Corrective Action: If 40 is reached, you will not be able to run any RMAN backups. In that case, bounce the instance.

DBA956 - "Server cron Daemon does not appear to be running - num_cron=$num_cron[$thishost]"

Description: There is no cron daemon owned by root running on this server.
Corrective Action: The cron daemon needs to be started. Own this ticket to the OS group.

DBAmon.com
This Document: http://dbamon.com/errors.shtml

Error Message Format

DBA005 - "DBAmon Governor Is Active"

DBA010 - "Logic Error - Invoked dbamon_orant.pl Did Not Finish"

DBA020 - "Error(s) Found in DBC File: $dbcfilename[$i]"

DBA101 - "$applhost - Missing $F3 process(es) - Number Found: $F5 - Should Be: $F6"

DBA101 - "$applhost - SAP Down - sapstart Process Not Running - Number Found: $F5 - Should Be: $F6"

DBA102 - "$applhost - SAP (R3check) Cannot Connect to DB"

DBA165 - "$applhost - /usr/sap/SID Filesystem is $bdfpct full ($bdfaval MB Free)"

DBA166 - "$applhost - Error running bdf /usr/sap/SID"

DBA209 - "No onbar/ontape archive data retrieved (No backup ever or DBAmon pgm. error)"

DBA210 - "(Backup Not Running) $rounded1-$rounded2 Hours since last good onarchive backup (threshold=$backup_age[$thishost])"

DBA211 - "$rounded1-$rounded2 Hours since last good onbar/ontape backup (threshold=$backup_age[$thishost])"

DBA212 - "ovtbls Value: $ovtbls Non-Zero"

DBA213 - "ovlock Value: $ovlock Non-Zero"

DBA214 - "ovuserthread Value: $ovuser Non-Zero"

DBA215 - "ovbuff Value: $ovbuff Non-Zero"

DBA216 - "Hours since last backup ($this_age) is a negative value"

DBA220 - "Critical Informix Message Log Message(s) Found"

DBA221 - "DBS Reporting Error: $error_text"

DBA222 - "TEMP DBSpace Full - DBS Probe Error"

DBA223 - "Informational Informix Message Log Message(s) Found"

DBA230 - "Total llog files=$num_llogs Full llog files=$full_llogs ($logpct percent)"

DBA231 - "Fatal dbaccess Error Checking Logical Logs"

DBA232 - "Fatal dbaccess Error Checking DBSpaces"

DBA233 - "Fatal dbaccess Error Checking Locks"

DBA234 - "Fatal Error running onstat -"

DBA236 - "$F2 (DBC Parm) File=$F3 Not Found -or- Not Readable"

DBA237 - "SQL Will Not Run"

DBA238 - "Informix Object(s) Found Whose Size Exceeds 25gB"

DBA239 - "ROOTPATH: $F2 Device File Is Not Readable"

DBA240 - "Read Hit Ratio $rh < Threshold of $t_readhit[$thishost]"

DBA241 - "Write Hit Ratio $wh < Threshold of $t_writehit[$thishost]"

DBA250 - "DBSpace $dbspace (${dbspct}% full ${dbsfree}MB free) exceeds critical threshold of $t_diskcrit[$thishost]"

DBA251 - "DBSpace $dbspace (${dbspct}% full ${dbsfree}MB free) exceeds warning threshold of $t_diskwarn[$thishost]"

DBA260 - "Informix Instance Not On-Line/Read-Only"

DBA261 - "Informix Instance Not On-Line/Read-Only"

DBA265 - "INFORMIXDIR Filesystem is $bdfpct full ($bdfaval MB Free)."

DBA266 - "Error running 'bdf \$INFORMIXDIR'"

DBA270 - "Table(s) found with >= 200 extents"

DBA271 - "Table(s) found with >= $max_extents[$thishost] extents"

DBA280 - "$offline_chunks Offline Chunk(s) Found"

DBA290 - "Connect Failure: ($conshort) After $remsh_try Attempt(s) - Server $pingable - Err=($remsherr) RC=$rc"

DBA291 - "Downloaded DBAmon Probe Software Not Found - Will retry download on next iteration"

DBA292 - "Connect resulted in compilation error"

DBA293 - "Probe Connect TIMEOUT (dbamonrc:Probe_Timeout=$probe_timeout)"

DBA299 - "Informix BUG Encountered (lockspct > 100) - Used Locks=$locksused Max Locks=$locksmax ($lockspct percent)"

DBA300 - "Used Locks=$locksused Max Locks=$locksmax ($lockspct percent)"

DBA301 - "ALL - DBAmon checking inhibited Date=$nowdate Hour=$nowhour"

DBA301 - "DBAmon checking inhibited Day=$nowday Monitor_Days=$mon_days[$h]"

DBA301 - "DBAmon checking inhibited DayHour=$dayhour Monitor_Excl=$mon_excl[$h]"

DBA301 - "DBAmon checking inhibited Hour=$nowhour Monitor_Hours=$mon_hours[$h]"

DBA301 - "DBAmon checking inhibited by ora_dbshut"

DBA301 - "This Server - DBAmon checking inhibited Date=$nowdate Hour=$nowhour"

DBA301 - "This Server_Instance - DBAmon checking inhibited Date=$nowdate Hour=$nowhour"

DBA302 - "FILECHECK File=$fc_file Not Found"

DBA303 - "FILECHECK File=$fc_file Is > 28 Hours Old"

DBA304 - "DBAmon checking inhibited by AUTO-INHIBIT"

DBA305 - "There are $userpw_cnt Non-System USERs whose userid=password"

DBA306 - "There are $userpwchanged_cnt System USERs whose userid=password - password changed"

DBA306 - "There are $userpwsys_cnt System USERs whose userid=password"

DBA308 - "DB is in ARCHIVELOG mode, but the archiver is STOPPED"

DBA309 - "Instance is HUNG waiting for ARCHIVER process to finish (All ONLINE REDO LOGS are full)"

DBA310 - "Instance has $shmvseg[$thishost] V shmem segments (critical threshold=$shmcrit)"

DBA311 - "Instance has $shmvseg[$thishost] V shmem segments (warning threshold=$shmcrit)"

DBA312 - "There are $dbarole_cnt Non-System USERs who have been granted the DBA role"

DBA313 - "There are $znum Possible Orphan Datafile(s) - $orphan_size gB (Threshold: 14 Days Since Last Update)"

DBA314 - "There are $sec_fileperm_cnt[$thishost] Oracle file(s) with incorrect permission(s)"

DBA315 - "There are $sec_dbarole_cnt[$thishost] Non-System USERs who have been granted the DBA role"

DBA316 - "The init.ora parm ? is (?) - It must be ? for security reasons"

DBA317 - "Tablespace(s) Found With NO Datafiles: $F3"

DBA318 - "There are 1 Non-System USERs who have been granted the ALTER USER priv"

DBA319 - "There are $dbarole_cnt Non-System USERs who have been granted the IMP_FULL_DATABASE role"

DBA320 - "DST2007 ($sw_version[$thishost]) JVM Is installed"

DBA321 - "DST2007 ($sw_version[$thishost]) SYS TZ Columns Founds"

DBA322 - "DST2007 ($sw_version[$thishost]) Non-SYS TZ Columns Found"

DBA323 - "DST2007 ($sw_version[$thishost]) TZ-Column/JVM Patch Required and Missing"

DBA330 - "CLEANERS ($cleaners[$thishost]) must be at least 75% of the number of disks ($num_disks[$thishost]) and >= (LRUS/2) (LRUS=$lrus[$thishost])"

DBA330 - "HDR Not Active - Type=$hdr_type State=$hdr_state Name=hdr_name"

DBA340 - "Duplicate ONCONFIG Parameters Found"

DBA350 - "Current Average Checkpoint Duration is $ckptavg[$thishost] Seconds"