We use LDAP users to install oracle software. In my humble opinion, it’s not a good approach because if the server can not communicate with LDAP service, Oracle gets errors when spawning new processes. We have already started to switch our oracle users from LDAP authentication to local users, but this is not the subject of this blog post. Using LDAP for authentication, also affects Enterprise Manager agents. When I try to create a named credential, EM agent can not verify the user/password, although I used the same user/password information to deploy the agents

Here’s the log generated by the agent:

[164049:8DB30DFF] INFO - >>> Reporting exception:
oracle.sysman.emSDK.agent.client.exception.PerformOperationException: 
ERROR: Invalid username and/or password   
LOG: Local Authentication Failed...Attempt PAM authentication...
PAM failed with error: Authentication failure (request id 1) <<<
oracle.sysman.emSDK.agent.client.exception.PerformOperationException: 
ERROR: Invalid username and/or password   
LOG: Local Authentication Failed...Attempt PAM authentication...
PAM failed with error: Authentication failure

The solution is well documented on My Oracle Support: How to Configure the Enterprise Management Agent Host Credentials for PAM and LDAP (Doc ID 422073.1)

In short, you need to create a file under /etc/pam.d directory named emagent i.e /etc/pam.d/emagent, and enter the following configuration lines into it:

#%PAM-1.0
auth     required pam_ldap.so
account  required pam_ldap.so
password required pam_ldap.so
session  required pam_ldap.so

I know that My Oracle Support is the most reliable source, but I decided to go my way! So instead of creating a new file containing the above configuration rules, I copied a working pam.d configuration file (/etc/pam.d/sshd) to “/etc/pam.d/emagent”. After I copied the configuration file, LDAP authentication started to working fine (no agent restart required on “Red Hat Enterprise Linux Server release 5.7 (Tikanga)”.

There was a question on OTN forums about how to generate alerts based on two different metrics. The user wants to get alert only if the warning threshold is over 80% “AND” there is less then 20Gb of free space of a tablespace. So he doesn’t want to get alert if the tablespace is over %80 full but still has 100GB free space. Of course, he can set different thresholds for each tablespace: He can set percentage threshold for small tablespaces and set free space threshold for bigger ones. I do not know how many databases he monitors but if he’s managing lots of DBs, this could be a time consuming task.

So how can we solve it? EM12c doesn’t let you generate an alert based on two different metrics. For these situations, you can create metric extensions. All you need is to query mgmt$alert_current and see if two alerts occurred for same target. Examine the following SQL:

SELECT targets.target_guid target_guid, key_value
 FROM   (SELECT target_guid, key_value,
                1 AS comp_alert
         FROM   mgmt$alert_current
         WHERE  metric_name = 'problemTbsp' and
            metric_column = 'bytesFree'
         INTERSECT
         SELECT target_guid, key_value,
                1 AS comp_alert
         FROM   mgmt$alert_current
         WHERE  metric_name = 'problemTbsp'
          and metric_column = 'pctUsed')  alerts,
         mgmt$target targets 
  WHERE  alerts.target_guid(+) = targets.target_guid
  GROUP BY targets.target_guid, key_value
  having Count(comp_alert) = 1;

This query checks if both “pctUsed” (Tablespace Space Used) and “bytesFree” (Tablespace Free Space) alerts are generated for same tablespace. The query doesn’t check the level of alert (warning or critical), but if you need so, you can use the alert_state column of mgmt$alert_current view.

Now let’s see how we can create it as a metric extension. First I have set the thresholds on one of my DBs (DEVDB) to produce alert for both percentage and disk space. So I will be able to test my metric extension.

Then I opened “metric extension” page and clicked “repository-side metric extension” under “create” menu. I will create a “repository-side metric extension”, so it will let me query management views of Enterprise Manager.

Because the tablespace metrics are related with databases, I pick “database instance” as the target type. When creating your own user metric, you can pick the target type according to your metrics.

I enter the above query, and click “next”. The query should always return target_guid and an additional column which Enterprise Manager will use as metric value.

At the columns page, we see one column returned from the query (target_guid is not visible here). I clicked on it to edit, and modified the display name, comparison operator and values for warning and critical. I entered a dummy value, because if my query returns any value (the name of the tablespace), I want this alert triggered! I also modified the messages to show which tablespace has low disk space.

We can check our metric before we actually deploy it. That’s a great feature! As you can see I added several instances, and run tests on them. Only one of them produced the alert.

I reviewed my metric extension and clicked “finish” to create it.

After it’s ready, all we need is to “save it as deployable draft”, and then deploy the targets.

This is an email generated by our metric extension. I hope it helps.

In another question on OTN forum, someone says that there are multiple (EM12c) users and they can create their own credentials. They ask how to view all of the credentials that exist for all users. A valid answer is already given: An EM administrator, including sysman, cannot view the named credentials owned by other administrators unless an explicit grant is provided.

I logged in as SYSMAN to Enterprise Manager Cloud Control console and checked the named credentials. As you can see, even SYSMAN can only see own credentials. So isnt’t there any way to override this “reasonable limitation”? I say “it’s reasonable”, because in my opinion, even SYSMAN (the Repository Owner) shouldn’t need to see the credentials of others. Anyway my curiosity wins, so I connected to repository Database (using Oracle SQL Developer) and examined tables, and I write the following query to show all named credentials:

SELECT cred_owner, cred_name, target_type, is_public 
FROM em_nc_creds
WHERE cred_owner <> '<SYSTEM>' 
ORDER BY cred_owner;

To be able to filter out credentials which are internally created by Enterprise Manager, I added a “where condition” to the query. Here’s the result:

So now you may wonder if it’s possible to get the passwords of these credentials? I’ll write about it on tomorrow

In my previous post, I have showed how to list all named credentials in Enterprise Manager Cloud Control. As you see, it was not possible using regular user interface, so we connected to the repository database to get the information. Now let’s keep digging and see if we can retrieve “encrypted information” saved in named credentials.

The username, password and role information of named credentials are stored in em_nc_cred_columns table. When we examine it, we can see that there’s one-to-many relation with em_nc_creds using target_guid column, and the sensitive information are stored in cred_attr_value column. That column is encrypted using em_crypto package. The encryption algrotim uses a secret key which is stored in “Admin Credentials Wallet” and a salt (random data for additional security). The wallet file is located in $MIDDLEWARE_HOME/gc_inst/em/EMGC_OMS1/sysman/config/adminCredsWallet/cwallet.sso) and the salt data can be found in cred_salt column of the em_nc_cred_columns table. Here’s what it looks like:

To decrypt the information, we need to call the decrypt in em_crypto package, but if we call it without opening the wallet, we get the following error:

ORA-06512: at line 1
28239. 00000 -  "no key provided"
*Cause:    A NULL value was passed in as an encryption or decryption key.
*Action:   Provide a non-NULL value for the key.

How can we read the secret key from that wallet? Easiest way is, make Enterprise Manager open the wallet and store the secret key in the repository database. So we issue the following command:

oracle@db-cloud ~$ /u02/Middleware/oms/bin/emctl config emkey -copy_to_repos
Oracle Enterprise Manager Cloud Control 12c Release 4
Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.
Enter Enterprise Manager Root (SYSMAN) Password :
The EMKey has been copied to the Management Repository. 
This operation will cause the EMKey to become unsecure.
After the required operation has been completed, 
secure the EMKey by running "emctl config emkey -remove_from_repos".

It asks for SYSMAN password. If you enter the correct password, it reads the wallet file and store the secret key into the repository database. Of course it makes your system unsecure. If you issue the command “emctl config emkey -remove_from_repos”, you can remove the key from repository.

Ok, if you issued the above command and stored the secret key into the repository, you can use the following query to fetch the decrypted information:

SELECT c.cred_owner,
c.cred_name,
c.target_type, 
(SELECT em_crypto.decrypt(p.cred_attr_value, p.cred_salt) 
FROM em_nc_cred_columns p WHERE c.cred_guid  = p.cred_guid 
AND lower(P.CRED_ATTR_NAME) LIKE '%user%') username,
(SELECT em_crypto.decrypt(p.cred_attr_value, p.cred_salt) 
FROM em_nc_cred_columns p WHERE c.cred_guid  = p.cred_guid 
AND lower(P.CRED_ATTR_NAME) LIKE '%role%') rolename,
(SELECT em_crypto.decrypt(p.cred_attr_value, p.cred_salt) 
FROM em_nc_cred_columns p WHERE c.cred_guid  = p.cred_guid 
AND lower(P.CRED_ATTR_NAME) LIKE '%password%') password
FROM em_nc_creds c
WHERE c.cred_owner <> '<SYSTEM>'
ORDER BY cred_owner;

Sample output:

As you may know, Weblogic is a part of Enterprise Manager Cloud Control environment, and it’s automatically installed and configured by the EM installer. The Enterprise Manager asks you to enter a username and password for Weblogic administration. This information is stored in secure files, and you usually do not need them unless you need to use Weblogic console. So it’s easy to forgot these username and password, and that’s what happened to me. Fortunately there’s a way to recover them without even resetting new user/password. Here are the steps:

First we need to know DOMAIN_HOME directory. My OMS is located in “/u02/Middleware/oms”. You can find yours if you read “/etc/oragchomelist”. If the full path of OMS is “/u02/Middleware/oms”, my middleware home is “/u02/Middleware/”. Under my middleware home, I need to go GCDomains folder:

oracle@db-cloud /$ cd /u02/Middleware
oracle@db-cloud Middleware$ cd gc_inst/user_projects/domains/GCDomain

First let’s get the encrypted information from boot.properties file:

oracle@db-cloud GCDomain$ cat servers/EMGC_ADMINSERVER/security/boot.properties

# Generated by Configuration Wizard on Wed Jun 04 10:22:47 EEST 2014
username={AES}nPuZvKIMjH4Ot2ZiiaSVT/RKbyBA6QITJE6ox56dHvk=
password={AES}krCf4h1du93tJOQcUg0QSoKamuNYYuGcAao1tFvHxzc=

The encrypted information starts with {AES} and ends with equal (=) sign. To decrypt the username and password, we will create a simple java application:

oracle@db-cloud GCDomain$ cat recoverpassword.java
public class recoverpassword {
 public static void main(String[] args)
 {
  System.out.println(
  new weblogic.security.internal.encryption.ClearOrEncryptedService(
  weblogic.security.internal.SerializedSystemIni.getEncryptionService(args[0]
   )).decrypt(args[1]));
  }
}

Save it as “recoverpassword.java”. To be able to compile (and run) it, we need to set environment variables (we’re still in GCDomain folder). We’ll give the encrypted part as the last parameter:

oracle@db-cloud GCDomain$ . bin/setDomainEnv.sh
oracle@db-cloud GCDomain$ javac recoverpassword.java
oracle@db-cloud GCDomain$ java -cp $CLASSPATH:. recoverpassword \
$DOMAIN_HOME {AES}nPuZvKIMjH4Ot2ZiiaSVT/RKbyBA6QITJE6ox56dHvk=
oracle@db-cloud GCDomain$ java -cp $CLASSPATH:. recoverpassword \
$DOMAIN_HOME {AES}krCf4h1du93tJOQcUg0QSoKamuNYYuGcAao1tFvHxzc=

I intentionally break the lines to make the code fits the page but you don’t need to do it. Correct CLASSPATH and DOMAIN_NAME are set when we issued “. bin/setDomainEnv.sh” command. When we run the last two commands, we should see the weblogic administrator username and password in plain text. By the way, even if you use the same password with me, you may see different encrypted text because when encrypting and decrypting, weblogic uses the cypher key stored in “security/SerializedSystemIni.dat” file. So as long as the cypher key is different, you get different encrypted text for even same input.

Java 1.8 is the current version and you probably use it on your personal computer. Although EMCLI says it will work on Java 1.6 and newer versions, if you install it using Java 1.8, you get an error when you try to setup or synchronize it with the Enterprise Manager Cloud Control OMS server. The reason is that Java 1.8 will use TLS 1.2 as default protocol for secure connections.

To fix the problem, you need to download and apply Patch 17555224 for FMW 11.1.0.6.0 version. You can learn more from My Oracle Support Doc ID 1930390.1.

EMCLI is not certified or supported on Mac OS X, but I wondered if it is possible to run on my new macbook. I searched metalink, googled and couldn’t found any useful information. I thought it should be easy because EMCLI is a java application certified to run almost all enterprise operating systems including HP-UX, AIX, Linux, Windows, Solaris, and we all remember “write once, run everywhere” slogan of Java. Unfortunately it wasn’t as easy as I expected

I have downloaded emclikit.jar from my Enterprise Manager web console, and tried to install it. Here’s the output:

gokhanatil$ java -jar emclikit.jar -installdir=emcli
Oracle Enterprise Manager 12c Release 4.
Copyright (c) 2012, 2014 Oracle Corporation.  All rights reserved.

Exception in thread "main" java.lang.UnsupportedOperationException
	at oracle.sysman.emSDK.emCLI.Utility.getFiles(Utility.java:640)
	at oracle.sysman.emCLI.CLICSInstaller.installClientSide(CLICSInstaller.java:35)
	at oracle.sysman.emCLI.CLICSInstaller.installClientSide(CLICSInstaller.java:28)
	at oracle.sysman.emCLI.CLICSInstaller.main(CLICSInstaller.java:21)
Problem while installing.

UnsupportedOperationException? What kind of unsupported operation could it be? For a Java application which its primary duty is to communicate with a web service, this error was surprising for me.

As you can see from the error stack, the exception was thrown in getFiles method. So I though there could be something about permissions of the files, or maybe something about how java access to folders. After some hours of debugging, I found that this exception was thrown intentionally when the emcli installer detects an unsupported operating system. So we hit an artificial barrier which is put there by the developer.

Is it possible to tell emcli installer that we use a supported OS (ie Solaris)? Not always, but most Java developers uses system property called “os.name” to detect the operating system, and our emcli developer also uses this method. We can add “-Dos.name=Solaris” parameter to fake the installer:

gokhanatil$ java -Dos.name=Solaris -jar emclikit.jar -install_dir=emcli2
Oracle Enterprise Manager 12c Release 4.
Copyright (c) 2012, 2014 Oracle Corporation.  All rights reserved.

Exception in thread "main" java.lang.UnsupportedOperationException
	at oracle.sysman.emSDK.emCLI.Utility.getFiles(Utility.java:640)
	at oracle.sysman.emCLI.CLICSInstaller.installClientSide(CLICSInstaller.java:35)
	at oracle.sysman.emCLI.CLICSInstaller.installClientSide(CLICSInstaller.java:28)
	at oracle.sysman.emCLI.CLICSInstaller.main(CLICSInstaller.java:21)
Problem while installing.

I like these kind of challenges It didn’t work because the jar file, calls another jaf file so our parameter doesn’t affect to the method. We need to change os.name permanently for our terminal session. The JAVA_TOOL_OPTIONS environment variable helps us to do it:

gokhanatil$ export JAVA_TOOL_OPTIONS=-Dos.name=Solaris
gokhanatil$ java -jar emclikit.jar -install_dir=emcli
Picked up JAVA_TOOL_OPTIONS: -Dos.name=Solaris
Oracle Enterprise Manager 12c Release 4.
Copyright (c) 2012, 2014 Oracle Corporation.  All rights reserved.

Picked up JAVA_TOOL_OPTIONS: -Dos.name=Solaris
EM CLI client-side install completed successfully.
Execute "emcli help setup" from the EM CLI home (the directory where you have installed 
EM CLI) for further instructions.

Hurray! We beat the detection. Have you noticed that two “Picked up JAVA_TOOLS_OPTIONS”? This is why the parameter “-Dos.name=Solaris” didn’t work. Anyway, because we have successfully installed the emcli, let’s run it:

gokhanatil$ ./emcli
Cannot find a java program of version "1.6" or greater.
Set the JAVA_HOME environment variable to a java "1.6" or greater home.

I use Java 1.8.x, so why does it say that it requires version 1.6 or greater? After examining the emcli script, I see that the developer finds the java version by running “java -version” and parsing the output. Unfortunately our output has an extra line:

gokhanatil$ java -version
Picked up JAVA_TOOL_OPTIONS: -Dos.name=Solaris
java version "1.8.0_31"
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)

The JAVA_TOOL_OPTIONS variable already served us well. We have installed the emcli, so I unset that variable, and include “-Dos.name=Solaris” into emcli script itself. Search for exec “$JAVA_PROGRAM” line in the file. You’ll find two. One for windows, one for Unix. You can easily understand it by reading the comment lines. Then add “-Dos.name=Solaris” after -DoracleHome=”$OracleHome”:

else
  # on unix

  # overriding the System property "user.name" is not needed on unix though
  # this would work for all unix-like platforms:
  #     EMCLI_USER=`id -un 2>/dev/null`
  # (for now, exact legacy behavior is maintained by not specifying user.name)
  #
  exec "$JAVA_PROGRAM" -Demcli.dir="$EMCLI_DIR"  -DoracleHome="$OracleHome" \ 
  -Dos.name=Solaris  -Djava.class.path="$EMCLI_CLASSPATH" \
  -Demcli.state.dir="$EMCLI_STATE_DIR" -Xmx512m $EMCLI_OPTS -jar "$EMCLI_JAR" "$@"
fi

After this change, I have successfully issued the following commands:

gokhanatil$ ./emcli setup -url="https://...:7802/em" -username=GOKHAN -trustall
Oracle Enterprise Manager 12c Release 4.
Copyright (c) 1996, 2014 Oracle Corporation and/or its affiliates. All rights reserved.

Enter password 

Emcli setup successful
gokhanatil$ ./emcli sync
Synchronized successfully

Everything looks OK! I haven’t tested all EMCLI verbs but for now it seems it works successfully. I also installed emcliadvancedkit (the one supporting Jpython scripting) with same method. Let me know if you encounter any problems. Of course, this is an unsupported and non-certified installation and you shouldn’t rely on it, and use it on production systems. You may wonder why we set the operating system variable to Solaris. There’s no reason. First I tried to install as setting os.name=Linux, spend some hours without success until I discovered JAVA_TOOL_OPTIONS variable, then I did set “os.name=Solaris”, and kept it as it is.

It’s for sure that Metric Extensions provide great flexibility to define your own metrics and enhance the monitoring abilities of Oracle Enterprise Manager Cloud Control. There are different adapters for different target types. You can see the available adapters when you pick a target at the “create new metric extension” page.

There are about 130 different target types, so I wondered if it’s possible to write a query to show available adapters for each target type.

SELECT DISTINCT t.type_display_name Target, 
  NVL(a.display_name,'SQL') adapter
FROM em_mext_adapter_props_def d, 
  em_mext_adapters a, 
  em_target_types t 
WHERE a.adapter_name = d.adapter_name 
AND t.target_type = d.target_type 
ORDER BY 1, 2;

After digging the tables of Enterprise Manager repository, I came up with the above query. Here’s the output:

If you’re wondering how you can use these adapters, you can get detailed information in Oracle Documents. By the way, the documentation is sufficient but I’m planning to write blog series to demonstrate how we can use each adapter type. So stay tuned!

Enterprise Manager Cloud Control 12c Release 5 is avaliable to download. The new release is designed to monitor and manage both your on-premise and Oracle Cloud deployments which is called “Hybrid Cloud” by Oracle. You can deploy Management Agents onto the Oracle Cloud virtual hosts to manage Oracle Cloud targets just as you would any other targets. EM12c R5 also supports data cloning between on-premise and Oracle Cloud. Integration with Oracle Enterprise Manager Ops Center will help dynamic on-demand resource provisioning.

I’m planning to upgrade our EM12c Release 4 to Release 5 in a couple of days, and discover the new features. Maybe I may write a couple of blog posts about upgrading or installing the new release.

As you may know, EM12c R5 was announced a couple of days ago, so I was regularly checking the download page, and noticed that EM12c never gets old, it is always NEW

Today, I tried to remove a plugin from Management Server but I see that undeployment process was hung at step 5:

----------------------------- ----------------------- -------------------------
Step                          Start Time              End Time                  
----------------------------- ----------------------- -------------------------
Submit job for undeployment   6/26/15 8:39:07 AM EEST 6/26/15 8:39:07 AM EEST Success

Initialize                    6/26/15 8:39:23 AM EEST 6/26/15 8:39:24 AM EEST Success

Validate plug-in home         6/26/15 8:39:25 AM EEST 6/26/15 8:39:27 AM EEST Success

Delete plug-in's metadata     6/26/15 8:39:28 AM EEST 6/26/15 8:39:49 AM EEST Success

Deconfigure plug-in from Management Repository 6/26/15 8:39:50 AM EEST N/A Running

----------------------------- ----------------------- -------------------------

I connected to Management Repository and see if there’s any waiting process. Unfortunately there was nothing. So I started to examine “active” processes of SYS user. You may know that EM12c uses SYS user when deploying or undeploying agent software. I found a process working for minutes, trying to complete the following commmand:

BEGIN SYSMAN.emd_maintenance.disable_em_dbms_jobs(); END;

It’s easy to guess that it’s related with disabling the scheduler jobs belong to Enterprise Manager but I wanted to be sure and check the package code. As expected, the procedure queries the scheduler jobs of the repository owner (SYSMAN) and then disable all its jobs but it hung. After a quick check on My Oracle Support, I couldn’t find any solution for emd_maintenance.disable_em_dbms_jobs so I decided to let him help by restarting the cjq0 process. I queried the job_queue_processes, set it to zero, and then reset it to its previous value.

SQL>  show parameter job

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
job_queue_processes                  integer     1000
SQL>  alter system set job_queue_processes=0;

System altered.

SQL>  alter system set job_queue_processes=1000;

System altered.

And then I see that undeployment process continued without any problem.

Scott Wardrop from AIDEV told me that they produced a plugin for Oracle Enterprise Manager Cloud Control 12c to monitor MongoDB. We use MongoDB in our production systems, so I decided to examine the plugin and write a blog post about it. Unfortunately version of our MongoDB databases are too old (2.4.x series) so the plugin didn’t work. Of course, it didn’t stop me. We created a simple MongoDB Cluster environment to test the MongoDB plugin for EM12c.

Becasue the installation is well documented by the vendor, I will not give a step by step installation guide. You can find the installation document on the download page. By the way, you need to manually download and deploy the plugin using EMCLI. Before you start deployment, send an email to info@aidev.uk for the unlock password (of the zip file). After you receive the password, download the latest plugin zip file, unzip it and run EMCLI:

cd /home/oracle
wget http://www.aidev.uk/12.1.0.3.0_aidev.mongo.xdbs_2000_0.zip
unzip 12.1.0.3.0_aidev.mongo.xdbs_2000_0.zip
emcli import_update -file=/home/oracle/12.1.0.3.0_aidev.mongo.xdbs_2000_0.opar -omslocal

After the installation is done, you need to deploy it to the OMS, and to the agents running on MongoDB servers.

Before you add the MongoDB targets using “Add Targets Declaratively by Specifying Target Monitoring Properties”, you also need to make some preparation on target MongoDB database and server. First you need to create a “monitoring” user. Login to target MongoDB database and run the following scripts:

use admin
db.createRole(
 {
 role: "emRole",
 privileges: [
{ resource: { db: "admin", collection: "system.users" }, actions: [ "find" ] },
{ resource: { db: "admin", collection: "system.roles" }, actions: [ "find" ] }
],
 roles: [ { role: "clusterMonitor", db: "admin" } ]
 }
)
db.createUser(
 {
 user: "em_monitor",
 pwd: "",
 roles: [ { role: "emRole", db: "admin" } ]
 }
)

The above script creates a new role (emRole) and give required permission to read-only access to MongoDB cluster/database, then it will create a new user and assign the emRole role to it. Don’t forget to modify the password. The plugin will need some JAR files, so download org.json libraries v20140107 and mongDB java driver v2.13.1 from http://search.maven.org/ to the target server(s). Be careful about the version numbers! The deployment document shows the correct versions and you should download exact versions. These files should be accessible by the user running EM agent.

When everything is ready, you can add the mongoDB database target. Click setup > add target > add target manually, pick the server, enter the target name, the user name (you created in MongoDB), the folder which contains jar files, MongoDB port number and then click OK. The plugin can monitor both mongos and mongod instances. If you’ll monitor a MongoDB cluster, I recommend you to add all nodes, config servers, database nodes and mongos instances to EM12c system.

If you connect mongos instance, you won’t be able to use “top waits” and “clients” page (these pages will be blank). On the other hand, to check the database sizes of the cluster, you need to connect mongos instances.

The MongoDB plugin shows very detailed information including replication info (when connected to database nodes), sharding information (when connected to mongos instances).

You can see how much disk storage used by each database, their growth history, connections, locks and many more. It is surely well designed and fetch all the useful information provided by MongoDB.

It’s also very useful to be able to see the logs of the instance.

One of the interesting feature of the plugin is, you can create javascript jobs to run on MongoDB servers. When testing this feature, I’ve found a small error about the path of the mongodb client. When creating a javascript job, instead of entering full path of the executable, you need to enter the path of the bin directory, because the plugin will add “mongo” to the path. I’m sure it will be fixed soon.

I liked the plugin, and I recommend you to use it if you want to monitor MongoDB clusters with EM12c. It’s really easy to connect Scott (the developer of the plugin) and ask questions, even ask new features By the way, I do not know the price but the plugin is not free. On the other hand, you can test it for 31 days at no cost.

Although EM12c R5 (12.1.0.5) was released about 2.5 months ago, I couldn’t find time to upgrade our EM12c system until now. I have decided to dedicate my day for upgrading our EM12c. I’m not planning to write a step by step document because it’s already documented well by other bloggers. I’ll just share my notes if I encounter any problem.

I have probably told several times that EM12c should have some internet access, at least, it should be able to reach My Oracle Support website. Luckily, our server has direct-access to the internet (not accepting any incoming connection requests), so I’ll download the installation files directly to the server. Because I like tricky things, I’ll use “wget” instead of an internet browser. I got a sample wget script from Oracle Support Website to download patches, and then modified it to download EM12c files. The script may be helpful if you’re planning to install EM12c to a remote server which you do not have X windows access.

As I said, the script is based on Oracle’s own wget script. I made only a few modifications. First, I needed to modify “–secure-protocol=auto” to “–secure-protocol=TLSv1”. Somehow, secure-protocol=auto didn’t work on our server. Then, I examined the download page of EM12c to find the URLs of installation zips (which belongs to Linux x86-64), and the cookie value which shows that we accept the license agreement (you know you should accept the license agreement to be able to download these files). I modified the script to use the cookie and fetch the correct files. Here’s the script:

#!/bin/sh

# SSO username and password
SSO_USERNAME=YOUREMAIL@YOURDOMAIN.COM
SSO_PASSWORD=YOURPASSWORD

# Path + options to wget command
WGET="/usr/bin/wget --secure-protocol=TLSv1"

# Location of cookie file
COOKIE_FILE=/tmp/$$.cookies

# Log file
LOGFILE=downloadem12c.log

# Contact updates site so that we can get SSO Params for logging in
SSO_RESPONSE=`$WGET --user-agent="Mozilla/5.0" "https://updates.oracle.com/Orion/Services/download" 2>&1|grep Location`

rm -f download

# Extract request parameters for SSO
SSO_TOKEN=`echo $SSO_RESPONSE| cut -d '=' -f 2|cut -d ' ' -f 1`
SSO_SERVER=`echo $SSO_RESPONSE| cut -d ' ' -f 2|cut -d 'p' -f 1,2`
SSO_AUTH_URL=sso/auth
AUTH_DATA="ssousername=$SSO_USERNAME&password=$SSO_PASSWORD&site2pstoretoken=$SSO_TOKEN"

# Login to Oracle Using SSO
$WGET --user-agent="Mozilla/5.0" --post-data $AUTH_DATA --save-cookies=$COOKIE_FILE --keep-session-cookies $SSO_SERVER$SSO_AUTH_URL -O sso.out >> $LOGFILE 2>&1

# Set Cookie to Accept License Agreement
echo ".oracle.com       TRUE    /       FALSE   0       oraclelicense   accept-gridcontrol_linx8664-cookie"     >> $COOKIE_FILE

rm -f sso.out

echo "Downloading em12105_linux64_disk1.zip"
$WGET --user-agent="Mozilla/5.0" --load-cookies=$COOKIE_FILE --save-cookies=$COOKIE_FILE --keep-session-cookies "http://download.oracle.com/otn/linux/oem/12105/em12105_linux64_disk1.zip" >> $LOGFILE 2>&1
echo "Downloading em12105_linux64_disk2.zip"
$WGET --user-agent="Mozilla/5.0" --load-cookies=$COOKIE_FILE --save-cookies=$COOKIE_FILE --keep-session-cookies "http://download.oracle.com/otn/linux/oem/12105/em12105_linux64_disk2.zip" >> $LOGFILE 2>&1
echo "Downloading em12105_linux64_disk3.zip"
$WGET --user-agent="Mozilla/5.0" --load-cookies=$COOKIE_FILE --save-cookies=$COOKIE_FILE --keep-session-cookies "http://download.oracle.com/otn/linux/oem/12105/em12105_linux64_disk3.zip" >> $LOGFILE 2>&1

Let me know if it works for you! By the way, you should enter your SSO login and password to be able to download the installation files.

Yesterday, I did my first Sybase IQ upgrade. I upgraded our main datawarehouse from Sybase IQ 16 to 16 SP10. The process was easy: Stop the database instance, install the new software, start the instance (from new software home), run an SQL to upgrade the metadata. Although the process was easy, I had some unexpected problems so I wanted to my notes in case you may need to upgrade your Sybase IQ.

You have probably heard it a thousand times but here I’ll say it again: Read the manual! Then open a ticket (or call) to Sybase support and ask questions about upgrading process. The process is very easy but it’s not documented well. Maybe I feel like this because I’m used to read Oracle’s word-class manuals. I’m serious. Oracle provides best manuals compared to the other Database vendors.

Second mandatory warning: Before you start to upgrade operation, make sure you have a valid backup of database.

Make sure that your license file is compatible with the software version you’ll install. You may need to generate an uptodate SySAM License! You can also contact Sybase suport and ask them to send you a compatible license file.

Sybase support send me a file named IQSERV160010P_2-20011180.zip. I have unzipped it and see that setup.bin (the installer) is not marked as executable. I made it executable and run it. That doesn’t sound like a big deal but later I figured out that while installing, the setup utility tries to execute other setup.bin files (located in subfolders), and none of them are marked as executable You may use such a one-line to mark them executable:

for f in `find . -name setup.bin`; do chmod +x $f; done

After updating software, check the version by running “start_iq -v2”. It will not start the instance, it will just show the software version. If you see the correct version (a higher version number than your previous one), start your instance, connect it using your favorite Sybase client, and run the following command:

ALTER DATABASE UPGRADE PROCEDURE ON

It will take only for a few seconds to modify metadata and your database is ready to go!

After upgrading software, somehow “Interactive SQL (dbisql)” tool started to not to work:

Interactive SQL could not load the "SQLAnywhere" plug-in.
Its "ngdbc.jar" file has moved or has been deleted. You will not be able to connect to the databases handled by that plug-in.
Interactive SQL could not load the "SybaseIQ" plug-in.
Its "ngdbc.jar" file has moved or has been deleted. You will not be able to connect to the databases handled by that plug-in.
Interactive SQL could not load the "HANA" plug-in.
Its "ngdbc.jar" file has moved or has been deleted. You will not be able to connect to the databases handled by that plug-in.
Interactive SQL could not load the "GenericODBC" plug-in.
Its "ngdbc.jar" file has moved or has been deleted. You will not be able to connect to the databases handled by that plug-in.
Interactive SQL cannot start because it is not installed correctly. No database plug-ins has been registered.
To fix this problem, you should reinstall the program.

To fix this problem, I re-registered IQ plugins to dbisql:

cd $SYBASE/IQ-16_0/java
dbisql -Xregister sa16 SybaseIQ com.sybase.saisqlplugin.IQISQLPlugin "$(pwd)/saip16.jar:$(pwd)/jodbc4.jar"

I’ll give presentations at DOAG 2015 on Tuesday (tomorrow), and at the BGOUG Autumn Conference on Saturday. My presentation will be about EMCLI (Enterprise Manager Command Line Interface). I will try to explain the basics of EMCLI, its terminology, the important points, and how we can use it with bash for scripting. I’ll also speak about EMCLI Python scripting, very short but useful into to Python and then share some sample scripts. Because of the time limit of the session, I need to be picky about sample scripts and put only a few ones (which points important points). I’ll share more sample scripts on my blog. The following script shows how you can write an EMCLI script which will read usernames, passwords and roles from a CSV file and then create these users as EM administrators. This script must be used with “EMCLI with Scripting Mode” (aka advancedkit):

#
# Sample EMCLI Python script file to create users
#

# check number of arguments
#
if len(sys.argv) <> 1:
    print "Usage: emcli @create_users.py filename.csv"
    exit()

# login to OMS
#
lg = login( username="SYSMAN" )

if lg.exit_code() <> 0:
    print lg.error()

# open file for reading
#
f = open( sys.argv[0], 'r' )

# loop for each line of the file
#
for line in f:
    # split line to list (array)
    #
    sp = line.split(',')
    #
    # create user, save the file name as description
    #
    create_user( name=sp[0], password=sp[1], roles=sp[2],
    desc="Listed in " + sys.argv[0])

I think the code is self-explanatory. As you can see, the script accepts only one argument (the name of the CSV file containing the information of new users. The arguments can be accessed via sys.argv object. Then it logins to OMS (it will ask the password of SYSMAN. You can modify the script and add your EM administrator username and password. At line 18, it opens the file, at line 24, it starts to loop for each line in the file. We split the string to an array at line 27, and use the members of the array to create new users at line 31.

Here’s the sample CSV file:

Joe,oracle,em_all_operator
William,oracle,EM_ALL_VIEWER
Jack,oracle,EM_CLOUD_ADMINISTRATOR
Averell,oracle,em_all_designer

By the way, I do not try to share “production quality” code. The sample code is just for guiding you to write better ones (for example: you may surely want to add exception handling etc…). If you have any questions about EMCLI Scripting, send them using contact form. I’ll be happy to answer them.

If you will attend to DOAG 2015 or BGOUG Autumn Conference, hope to see you in my session:
 

Jump into the EMCLI: A Crash Course for EM12c Administrators
Date: 2015-11-17 Time: 4:00 pm Room: Tokio
 

Jump into the EMCLI: A Crash Course for EM12c Administrators
Date: 2015-11-21 Time: 10:00 am Room: Hall B

After the presentation about EMCLI at DOAG 2015, someone asked me how we can list database targets based on their operating systems. In my presentation, I told that there are more than 70 verbs which starts with “get_” and help you fetch information about EM objects, and if you can’t find what you’re looking for, you can use the “list” verb which is probably the most powerful verb of EMCLI. Now I try to show how we can use the list verb to fetch detailed information about EM targets.

Let’s say we want to list the databases running on Linux. The original question was how we can list the databases running on Windows but thanks God, I don’t have any databases running on Windows. So I changed the question. First let’s see what we can get by using “get_targets”:

emcli get_targets -target=”oracle_database”:

As we can see, there’s no information about the operating systems. There’s not even any information about which host these databases are running on. Now let’s try the “list verb” for any of the target databases such as DEVDB:

emcli list -resource="Targets" -search="TARGET_NAME='DEVDB'" -format="name:csv"
TARGET_NAME,TARGET_TYPE,TARGET_GUID,TYPE_VERSION,TYPE_QUALIFIER1,TYPE_QUALIFIER2,
TYPE_QUALIFIER3,TYPE_QUALIFIER4,TYPE_QUALIFIER5,EMD_URL,TIMEZONE_REGION,DISPLAY_NAME,
HOST_NAME,LAST_METRIC_LOAD_TIME,TYPE_DISPLAY_NAME,BROKEN_REASON,BROKEN_STR,OWNER,
LAST_LOAD_TIME_UTC,CREATION_DATE
DEVDB,oracle_database,102890E057665EC4E053E03210AC4FA3,5.4,11gR202,2,DB,,FullLLFile,
https://db03-test.bilyoner.com:3872/emd/main/,Europe/Istanbul,
DEVDB,db03-test.bilyoner.com,2015-11-12 10:28:55.0,Database Instance,0,,SYSMAN,
2015-11-12 08:28:55.0,2015-02-28 17:49:51.0

Using “list” verb, we found the host name. Now we can use the same command to get more information for that host:

emcli list -resource="Targets" -search="TARGET_NAME='db03-test.bilyoner.com'"
TARGET_NAME,TARGET_TYPE,TARGET_GUID,TYPE_VERSION,TYPE_QUALIFIER1,TYPE_QUALIFIER2,
TYPE_QUALIFIER3,TYPE_QUALIFIER4,TYPE_QUALIFIER5,EMD_URL,TIMEZONE_REGION,
DISPLAY_NAME,HOST_NAME,LAST_METRIC_LOAD_TIME,TYPE_DISPLAY_NAME,BROKEN_REASON,
BROKEN_STR,OWNER,LAST_LOAD_TIME_UTC,CREATION_DATE
db03-test.bilyoner.com,host,FCBE4FFC339F6CA7E043E03210ACE0D9,4.5,Linux,
Red Hat Enterprise Linux Server release 5.11 (Tikanga),Generic,,,
https://db03-test.bilyoner.com:3872/emd/main/,Europe/Istanbul,db03-test.bilyoner.com,
db03-test.bilyoner.com,2015-11-12 10:15:41.0,Host,0,,SYSMAN,2015-11-12 08:15:41.0,
2014-06-26 16:40:17.0

The “TYPE_QUALIFIER1” column holds the name of OS if the target is a host. It holds the database version if the target is an oracle database. Unfortunately there’s not an easy way to combine this information when querying the “resources”. So we need to get all oracle databases and then get the host information for each database. Of course, it won’t be a practical approach but here’s a sample code to do it using EMCLI Python (Jython) scripting engine:

if (len(sys.argv) <> 1 ):
    print "Usage: emcli @list_targets.py OSname"
    exit()

login( username="EMuser", password="EMpassword")

myDBs = list( resource="Targets", search="TARGET_TYPE='oracle_database'" )
             
for DB in myDBs.out()['data']:
    # we get the host info for the oracle database target
    hostinfo = list( resource="Targets",
    search="TARGET_NAME='" + DB['HOST_NAME'] + "'" )
    # the above one should return only one item, so we use the first item
    OS = hostinfo.out()['data'][0]['TYPE_QUALIFIER1']
    # we print only the targets if their OS name equal to the first argument of script
    if ( OS.lower() == sys.argv[0].lower() ):
        print DB['TARGET_NAME'],hostinfo.out()['data'][0]['TYPE_QUALIFIER1']

Save the above code as list_targets.py, then run it:

We got the desired result but the method we used is not efficient. Let’s try to get the same info using a different method. The list verb can also be used to execute SQL on EM repository. It will login as MGMT_VIEW, and run the query that you give in “-sql” parameter. So instead of querying “predefined” resources, we can directly query MGMT$TARGET view:

emcli list -sql="select t.target_name DBANME, os.TYPE_QUALIFIER1 OSNAME from 
MGMT\$TARGET t, MGMT\$TARGET os where t.HOST_NAME = os.TARGET_NAME 
and t.target_type='oracle_database' and os.TYPE_QUALIFIER1 = 'Linux'"

And this is the result:

We get the desired result with a simple EMCLI command. It’s much faster and cleaner.

Last week, I gave presentations at DOAG (German Oracle User Group) conference and BGOUG (Bulgarian Oracle User Group) autumn conference about Oracle Enterprise Manager Command Line Interface. It was my first time at these conferences, and both of them were wonderful.

Before each session at DOAG Conference, there was a video showing “how they prepared the last year’s DOAG conference”. What a professionalism! What a great team work! You know Germany is known for their high quality standards, and you could see it everywhere at DOAG conference. I feel privileged to speak there. I had chance to meet lots of people I know from Twitter, face to face.

I was very lucky to join to the ACE Dinner which was organized for the first time at DOAG!

BGOUG Autumn Conference was warm and friendly. It was held in Riu Hotels at Pravets. Pravets is a lovely small town close to Sofia. The scenery, the hotel, the attendees were all great. Unfortunately my session was at early hours right after the day we had party (ah yes, there was a conference Party and it was also great), not so much people showed up. I still admired how BGOUG organized such a great event. I hope we’ll organize such events as TROUG (Turkish Oracle User Group).

Anyway, this post is not about how much I enjoyed from being at these conferences (although I did). I have uploaded the my presentation to slideshare. Please take a look, and if you have any questions, do not hesitate to ask.

Hope to see you in another event.

You have probably heard that Oracle released Enterprise Manager Cloud Control 13c on Friday. It’s the first product from Oracle with the number 13 in its release label. It seems Oracle doesn’t care about the bad luck of 13 The new version comes with great features and maybe the most significant one is the unification of hardware and software management. In Oracle Enterprise Manager 13c, some of the important hardware management features of “Ops Center” have been included into the Cloud Control product. My company is not a beta tester of Enterprise Manager, so I wanted to upgrade our EM12c system as soon as possible to test the new features by myself. I upgraded our EM12c system yesterday. Well, as I expected, the upgrade process was easy but you should be careful about prerequisites. Carefully check the certification matrix (in My Oracle Support website) before you decide to upgrade your system! EM13c OMS requires Oracle 12.1.0.2 as repository database, and EM13c agents don’t support Enterprise Linux earlier than version 6. We have redhat Enterprise Linux 5.x targets and our repository database was 11.2.0.4 so I have upgraded our repository database and decided to use the “old agents” with EM13c until we upgrade our servers to Enterprise Linux 6.x.

Please take a look at Enterprise Manager Cloud Control Upgrade Guide, before you start upgrading and make sure that you have backed up the repository database and OMS! I’ll try to provide a step by step guide on upgrading EM12c to EM13c running on Linux system, but this is just to give you an idea about overall process, you should always read and follow the official guide.

Important addition from Sumesh: If you have the below obsolete plug-ins in your EM12.1.0.3/4/5 system, then undeploy them from OMS and Agent before proceeding with upgrade to EM13.1:

• EMC Symmetrix Array (oracle.em.sesy)
• EMC Celerra Storage (oracle.em.secs)
• EMC CLARiiON Array (oracle.em.secl)
• Microsoft Biz Talk Server (oracle.em.smbs)
• Exalogic Elastic Cloud Infrastructure (oracle.em.soee)

Make sure the Repopsitory Database and OMS machine are in the same time zone. Eg. UTC or PDT. If the timezone is not same then, Upgrade will fail at ‘Plug-ins Prequisites check’ step.

When upgrading, EM13c will need to access EMKey (the secret key to encrypt password etc), so we copy the EMKey to the Management Repository:

$OMS_HOME/bin/emctl config emkey -copy_to_repos

We also need to disable the optimizer adaptive feature. So we connect to our repository database and run the following SQL command:

alter system set optimizer_adaptive_features=false scope=spfile;

If you have upgraded your database like, you also need to make sure that database compatibility parameter equal to the database version:

alter system set compatible="12.1.0.2.0" scope=spfile

Then restart the repository database, to make the new changes have taken effect.

Download the installation files from Oracle web site, and run cksum to verify checksums and sizes of the files. EM13c installation doesn’t need you to unzip the files. All you need is to put all files into the same directory, mark em13100_linux64.bin as executable and run it with “oracle” user (the user you installed your current Enterprise Manager):

chmod +x em13100_linux64.bin
./em13100_linux64.bin

You’ll see a familiar screen which asks you to enter your email. Enter your email (or just leave it blank), and then click next button.

Because it’s just released there’s no available patches for EM13c right now, but I still wanted to be sure. Click next to continue.

Examine the list, and if there are important failures, fix them before you continue to upgrade. In the prerequisites check, “ip local port range” gives warning on my system because it’s set to wider range, so I have just decided to ignore the warning.

As you can see, there’s only “one system” upgrade option, so we pick it and continue.

We enter the new middleware location of EM13c.

Enter the SYS and SYSMAN passwords of the repository database. Deferred Data Migration (DDMP) is a post-upgrade activity to migrate the historical data of your old Enterprise Manager. Depending on how much data stored, it may take same time. So you may prefer to run the DDMP jobs later from the Post Upgrade Console.

You may get some warnings about the database, some of them can be fixed automatically by the installer (when you click YES), some of them need manual fixes. Fix them and continue.

Read the repost about the plugin upgrade, click next to continue.

Select additional plugins if you’re planning to use them. If you haven’t stopped the OMS, it’s time to stop it now. Run the following command in the terminal windows and then click next on your installer:

$OMS_HOME/bin/emctl stop oms -all

You also need to stop the agents running on the repository server and management servers! Go to the agent installation directory and run the following:

$AGENT_HOME/agent_inst/bin/emctl stop agent

Although you stop the agents, you may still get a warning about them. Ignore it and continue to upgrade.

Enter the weblogic password, and enter the new location for OMS instance. If you forgot the password, you can recover it. EM13c wants you to choose a directory outside the middleware home.

EM13c comes with BI Publisher. In our existing Enterprise Manager system, we haven’t configured BI Publisher, so I entered directory names to configure it. If you are upgrading an OMS that already has Oracle BI Publisher installed, then these fields should be prefilled and grayed out.

I accept the default ports and click next to continue.

Review the information you entered, and then click the upgrade button to start upgrading.

It took more than one hour on my system but I’m aware that our disks are a little bit slow so it may take less time on a better hardware.

As usual, you see a message box asking you to run “a root script”. It’s in middleware home. After you run the script (with root user), click OK and you’ll see the above screen saying the upgrade is completed.

Now EM13c is ready and waiting you to login After you login, go to the “upgrade agents” page under the “setup menu” to upgrade your agents. If you have disabled DDMP jobs, you should also go to “post upgrade tasks” under the “setup menu” and start deferred data migration tasks.

When I was trying to discover the new features of EM13c, I noticed that EM13c provides you a new console called the Initial Setup Console. If you’re new to Enterprise Manager, you may wonder if you configured EM properly. The initial setup console is a great starting point to set up Enterprise Manager Cloud Control. You can reach this console under the setup menu.

It will help you to perform the following tasks:

• Configuring Oracle Software Library Using the Initial Setup Console to Store Software Entities
• Configuring Proxies for OMS-to-Management Agent Communication
• Configuring Proxies Using the Initial Setup Console for OMS-My Oracle Support Communication
• Adding Additional Oracle Management Service Using the Initial Setup Console
• Configuring Outgoing Mail Servers (SMTP Servers) Using the Initial Setup Console
• Registering My Oracle Support Credentials Using the Initial Setup Console
• Creating Users Using the Initial Setup Console
• Creating Roles Using the Initial Setup Console

You don’t need to complete all tasks. You can ignore any task, and complete them whenever you want (for example adding additional OMS). It doesn’t seem like an important feature but I think it will be really helpful you to configure your Enterprise Manager Cloud Control environment. By the way, this console is only available to super administrators.

Sending notification emails shouldn’t require any licenses right? Unfortunately it might be more complicated than it seems

As you know Enterprise Manager Cloud Control provides a web interface called web console, and (naturally) it consists of web pages that enables you to access the features of Enterprise Manager Framework. Some of these features (mostly the basic ones) are free but some of them require you to buy management pack licenses. When you’re not sure if the page you accessed requires a management pack license, you can check it by clicking “Packs for this Page” (setup > management packs). It will show you a popup and list the required management packs. It’s also possible to enable annotations. When annotations are enabled, Enterprise Manager indicates licensed links by showing an abbreviation for a pack beside the licensed link. I noticed that there’s an interesting problem with using the notification feature of Enterprise Manager. In EM12c, when you check the required licenses for the notification feature/page, you’ll see that it requires almost all management packages. And according to the My Oracle Support Doc ID 1936531.1, the required management packs for notifications feature are listed correctly!

In my humble opinion, the notifications feature should not require any management pack licenses as long as it will be used for sending notifications for basic monitoring and incident handling. The interesting thing is, in EM13c, some of the notifications feature pages are divided to multiple pages and it seems basic notification features do not need management pack licenses anymore.

So in EM13c, if you don’t customize notification emails or notification schedule, and don’t use SNMPv1 traps, the notifications feature doesn’t require management pack licenses. I hope someone from Oracle confirms that the notifications feature doesn’t need any of these licenses at all. Or at least, the notifications feature of EM12c is licensed same to EM13c.

After we upgraded EM13c, I encountered a problem about automatically promoted listener targets. Since EM12c, I do not add/promote listeners. In earlier versions of EM12c, I had weird problems while monitoring listeners on RAC systems (such as they are detected down although they’re up). Since then, I don’t monitor them (specially SCAN listeners). In EM12c, when I remove a listener, it doesn’t promoted again. With EM13c, I see that listeners are discovered and promoted automatically on each day. First I though, one of my team members accidentally add them, but then I see that it’s an automatic process. After a quick search on MoS, I didn’t find anything and opened a service request. After several mails, Oracle support staff told me that it’s an expected behavior, and I need to disable “autodiscovery” to prevent it (Doc ID 1522674.1) – I really don’t know how I couldn’t find it by myself, but even Oracle support guys spent 3 weeks to find the note

So if you don’t want your listeners and other clusterware managed targets are discovered and promoted continuesly, you need to disable discovery for these targets:

To disable automatic addition of RAC instance targets:

emctl set property -name oracle.sysman.db.discovery.eventBased.racInstAdd -value false

To disable automatic addition of Listener targets:

emctl set property -name oracle.sysman.db.discovery.eventBased.listenerAdd -value false

To disable automatic addition of ASM instance targets:

emctl set property -name oracle.sysman.db.discovery.eventBased.asmInstAdd -value false

To disable automatic addition of Host targets to Cluster targets:

emctl set property \
-name oracle.sysman.db.discovery.eventBased.hostToClusterAdd -value false

To disable automatic promotion of HAS targets:

emctl set property -name oracle.sysman.db.discovery.eventBased.hasPromote -value false

You need to run these commands on the OMS server (it’s enough to execute them on one of the OMS servers). After you change the property, all the OMS instances must be restarted for the parameters to be active!