EM12c Adding Targets and Keeping it Clean

I often have folks ask me for assistance when target discovery isn’t successful.  The following is from a client’s environment that shows just how important it is to ensure your server environment is kept pristine.

The goal was to discover a new database on an existing RAC cluster.  The cluster had already been discovered and configured, but the DBA was experiencing a failure upon adding the database targets.  Once the cluster is configured through a manual discovery using the GUID wizard, adding a RAC database is often a simple process.  Just click on Targets and Databases from the EM12c console and you are on your way.

 

You can then click on the ADD button to start adding a database target.

The wizard will ask you to choose a host, which for a RAC environment, choose any one of the server nodes from the cluster to proceed.

Click on Continue to proceed and Enterprise Manager will inspect each host that is part of the cluster and ask you how you would like to proceed:

Unless there is a reason to ignore one or more nodes, choose to inspect all hosts that are part of the cluster and click continue.

Now this is where our story takes a bad turn.  The discovery proceeded to check through each of the targets that made up the cluster.  The wizard actually shows you each of the host names as it works through the list of host targets.

After it completed it’s discovery, no databases were discovered and discovery warnings were shown.

The databases they expected to show up didn’t and the list then included one called XTTS, but there were also errors in the discovery shown at the top.  Clicking on the View Discovery Warnings shows the following:

This error may not make a lot of sense when first looking at it, but the error actually tells you all you need to know about the problem at hand.

Steps to add targets from any host target/node target include parsing the /etc/oratab file, (on Linux, as least.  For other OS, the oratab file exists in different locations and for Windows, EM12c will check the services.)  The error even tells you which target node experienced the problem.  With this information in hand, log into the host that is identified as having the error and inspect the oratab file.

Note upon viewing this file, (I’ve blacked out the entries for the client’s production databases) there are four discrepancies once we do a quick check.

• Why is there an entry for ‘ASM’ and another for the correct ‘+ASM’?
• There isn’t a SID named db.
• There isn’t a SID xtts
• There isn’t an ORACLE_HOME /u01/app/oracle/product/1102_xtts

The fourth reason is the cause for the odd error in the discovery and the other incorrect entries are impacting the discovery process from locating databases properly.  If we look at the other nodes, none of them have these “bogus” entries in the oratab file, (although they could benefit from a bit of cleanup, too!):

(Again, I’ve hidden the production databases and only the agent entries are showing)

I proceeded clean up the following entries from ALL of the files:

• ASM entries, (vs. +ASM)
• XTTS instance
• DB instance
• Any 11g agent, as the client now only used the EM12c

I then saved the changes to the oratab file and started the discovery over.  Success and no longer does the error occur.  The importance of keeping your support files on each database server cleaned up is crucial to the Enterprise Manager 12c for accurate target discovery.

Have a great weekend!