Category: Oracle

April 27th, 2016 by dbakevlar

Someone pinged me earlier today and said, “Do I even really need to know about logs in Enterprise Manager?  I mean, it’s a GUI, (graphical user interface) so the logs should be unnecessary to the administrator.”

luke

You just explained why we receive so many emails from database experts stuck on issues with EM, thinking its “just a GUI”.

Log Files

Yes, there are a lot of logs involved with the Enterprise Manager.  With the introduction back in EM10g of the agent, there were more and with the EM11g, the weblogic tier, we added more.  EM12c added functionality never dreamed before and with it, MORE logs, but don’t dispair, because we’ve also tried to streamline those logs and where we weren’t able to streamline, we at least came up with a directory path naming convention that eased you from having to search for information so often.

The directory structure for the most important EM logs are in the $OMS_HOME/gc_inst/em/OMGC_OMS1/sysman/log directory.

Now in many threads on Oracle Support and in blogs, you’ll hear about the emctl.log, but today I’m going to spend some time on the emoms properties, trace and log files.  Now the EMOMS naming convention is just what you would think it’s about-  the Enterprise Manager Oracle Management Service, aka EMOMS.

The PROPERTIES File

After all that talk about logs, we’re going to jump into the configuration files first.  The emoms.properties is in a couple directory locations over in the $OMS_HOME/gc_inst/em/EMGC_OMS1/sysman/config directory.

Now in EM12c, this file, along with the emomslogging.properties file was very important to the configuration of the OMS and it’s logging, which without this, we wouldn’t have any trace or log files or at least the OMS wouldn’t know what to do with the output data it collected!  If you look in the emoms.properties/emomslogging.properties files for EM13c, you’ll receive the following header:

#NOTE
#----
#1. EMOMS(LOGGING).PROPERTIES FILE HAS BEEN REMOVED

Yes, the file is simply a place holder and you now use EMCTL commands to configure the OMS and logging properties.

There are, actually, very helpful commands listed in the property file to tell you HOW to update your EM OMS properties!  Know if you can’t remember an emctl property commands, this is a good place to look to find the command/usage.

The TRACE Files

Trace files are recognized by any DBA-  These files trace a process and for the emoms*.trc files, these are the trace files for EM OMS processes, including the one for the Oracle Management Service.  Know that a “warning” isn’t always a thing to be concerned about.  Sometimes it’s just letting you know what’s going on in the system, (yeah, I know, shouldn’t they just classify that INFO then?”

2016-04-09 01:00:07,523 [RJob Step 62480] WARN jobCommand.JvmdHealthReportJob logp.251 - JVMD Health report job has started

These files do contain more information than the standard log file, but it may be more than what a standard EM administrator is going to search through.  They’re most helpful when working with MOS and I recommend uploading the corresponding trace files if there is a log that support has narrowed in on.

The LOG Files

Most of the time, you’re going to be in this directory, looking at the emctl.og, but remember that the emoms.log is there for research as well.  If you perform any task that involves the OMS and an error occurs, it should be written to the emoms.log, so looking at this log can provide insight to the issue you’re investigating.

The format of the logs are important to understand and I know I’ve blogged about this in the past, but we’ll just do a quick and high level review.  Taking the following entry:

2016-01-12 14:54:56,702 [[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'] ERROR deploymentservice.OMSInfo logp.251 - Failed to get all oms info

We can see that the log entry starts with timestamp, module, message, status, (ERROR, WARN, INFO) detail, error message.  This simplifies it when having to read these logs or knowing how one would parse them into a log analysis program.

There are other emoms log files, simply specializing in loader processing and startup.  Each of these logs commonly contain a log file with more detailed information about the data its in charge of tracing.

If you want to learn more, I’d recommend reading up on EM logging from Oracle.

 

 

Posted in EM13c, Enterprise Manager, Oracle Tagged with: , ,

April 20th, 2016 by dbakevlar

How much do you know about the big push to BI Publisher reports from Information Publisher reporting in Enterprise Manager 13c?  Be honest now, Pete Sharman is watching…. 🙂

george

I promise, there won’t be a quiz at the end of this post, but its important for everyone to start recognizing the power behind the new reporting strategy.  Pete was the PM over the big push in EM13c and has a great blog post with numerous resource links, so I’ll leave him to quizzing everyone!

IP Reports are incredibly powerful and I don’t see them going away soon, but they have a lot of limitations, too.  With the “harder” push to BI Publisher with EM13c, users receive a more robust reporting platform that is able to support the functionality that is required of an IT Infrastructure tool.

BI Publisher

You can access the BI Publisher in EM13c from the Enterprise drop down menu-

bippub4

There’s a plethora of reports already built out for you to utilize!  These reports access only the OMR, (Oracle EM Management Repository) and cover numerous categories:

  • Target information and status
  • Cloud
  • Security
  • Resource and consolidation planning
  • Metrics, incidents and alerting

bipub3

Note: Please be aware that the license for BI Publisher included with Enterprise Manager only covers reporting against the OMR and not any other targets DIRECTLY.  If you decide to build reports against data residing in targets outside the repository, it will need to be licensed for each.

Many of the original reports that were converted over from IP Reports were done so by a wonderful Oracle partner, Blue Medora, who are well known for their VMware plugins for Enterprise Manager.

BI Publisher Interface

Once you click on one of the reports, you’ll be taken from the EM13c interface to the BI Publisher one.  Don’t panic when that screen changes-  it’s supposed to do that.

bipub4

 

You’ll notice be brought to the Home page, but you’ll notice that you’ll have access to your catalog of reports, (it will mirror the reports in the EM13c reporting interface) the ability to create New reports, open reports that you may have drafts of or are local to your machine, (not uploaded to the repository) and authentication information.

In the left hand side bar, you will have menu options that duplicate some of what is in the top menu and tips access to help you get more acquainted with BI Publisher-

bipub7

This is where you’ll most likely access the catalog, create reports and download local BIP tools to use on your desktop.

Running Standard Reports

 

To run a standard, pre-created report, is pretty easy.  This is a report that’s already had the template format created for you and the data sources linked.  Oracle has tried to create a number of reports in categories it thought most IT departments would need, but let’s just run two to demonstrate.

Let’s say you want to know about Database Group Health.  Now there’s not a lot connected to my small development environment, (four databases, three in the Oracle Public Cloud and one on-premise) and this is currently aimed at my EM repository. This limits the results, but as you can see, it shows the current availability, the current number of incidents and compliance violations.bipub1

We could also take a look at what kinds of targets exist in the Enterprise Manager environment:

bipub11

Or who has powerful privileges in the environment:

bipub10

Now this is just a couple of the dozens of reports available to you that can be run, copied, edited and sourced for your own environment’s reporting needs out of the BI Publisher.    I’d definitely recommend that if you haven’t checked out BI Publisher, spend a little time on it and see how much it can do!

 

 

Posted in EM13c, Enterprise Manager, Oracle Tagged with: , ,

April 7th, 2016 by dbakevlar

It’s that time of year again and the massive undertaking of the Collaborate conference is upon us.  This yearly conference, a collaboration between Quest, Oracle Applications User Group, (OAUG) and Independent Oracle User Group, (IOUG) is one of the largest conferences in the world for those that specialize in all areas of the Oracle database.

The conferene is held in different cities, but recently its been sticking to the great destination of Las Vegas, NV.  We’ll be at the Mandalay, which like many casinos, is like it’s own little self-contained city within a city.

vegasbaby

The week will be crazy and I’ll start right out, quite busy with my partners in crime, Courtney Llamas and Werner De Gruyter with Sunday’s pre-conference hands on lab. “Everything I Ever Wanted to Know About Enterprise Manager I Learned at Collaborate” was a huge hit last year, so we’re repeating it this year, but we’ve updated it to the newest release, EM13c.  For those that are able to gain a coveted spot in this HOL, it will be a choice event.  We’re going to not just cover the new user interface, but some of the coolest need-to-know features of the new release.

Sunday evening is the Welcome Reception and Awards Ceremony.  This year I’m receiving the Ken Jacobs award for my contributions to the user community as an Oracle employee.  I’m very honored to be receiving this and thank everyone at IOUG for recognizing the importance that even as an Oracle employee, you can do a lot to help make the community great!

Throughout the week, I’ll have a number of technical sessions:

Monday

Now my Database as a Service Session is up first for the week on Monday, 9:15am in Palm B, but I’m going to warn you, since this abstract was submitted very early on, the abstract isn’t as descriptive as I wanted.  Know that this is a DBaaS session and I’ll be covering on-premise, private cloud and even Oracle Public Cloud!  Come learn how easy it can be and forget all those datapump, transportable tablespace and other silly commands people are telling you have to do to provision… ☺

Right  after my DBaaS session, 10:15, same room, (Palm B) we’ll have a special session covering the new product that so many of us have put so much energy, time and vision into-  The Oracle Management Cloud, (OMC)!  Read more about this session here.

The Welcome Reception in the Exhibit Hall is from 5:30-8pm.  Don’t miss out on getting there first and see all the cool exhibitors.  I’ll be at the EM13c booth, so come say hi!

Tuesday

So Tuesday morning, the 12th, I’m back in Palm B at noon for the first of my certification sessions, covering 30 minutes of Enterprise Manager 13c New Features.

Wednesday

Wednesday, at noon, I’m back in my favorite room, Palm B to finish the second part of the certification sessions on new features with Enterprise Manager 13c.

I’ll be presenting at Oak Table World at Collaborate at 2pm in the Mandalay Bay Ballroom.  I’ll be doing my newest session on Enterprise Manager 13c and DB12c.  It’s always a great venue when we have Oakies at conferences and I almost squeaked out of it this year, but dragged back in at the last minute!

The Enterprise Manager SIG is right afterwards at 3-4  in the South Seas Ballroom E.  This is where we meet and geek out over everything Enterprise Manager, so don’t miss out on that!

Thursday

For the last day, Thursday at 9:45am, I’ll be in- wait for it….  Palm B!  Yes, I know it’s a surprise for both of us, but I’ll be using my experience helping customers Upgrade to Enterprise Manager 13c and sharing it with everyone at Collaborate.  This is another certification session, so collect those certificates and get the most out of your conference!

I’ve made a lot of updates with new material to my slides recently, so I promise to upload my slides to SlideShare after the conference, too!

See you next week in Las Vegas!

 

 

Posted in ASH and AWR, EM13c, Oracle, Oracle Management Cloud Tagged with: , , , ,

March 28th, 2016 by dbakevlar

Even though I didn’t have the “official” prerequisite classes of HTML and CSS for the JavaScript class offered by sister Meetup group, Girls Develop it, I decided on Friday that I wanted to take the weekend class and signed up.

The class is held at the Turing Development school and it was a great downtown location.  Very centralized, no matter if you’re North, South, East or West of the city and the venue is a school, so it’s set up with plenty of power, WiFi and projector with multiple screens.  It had started another spring time snow, so I was one of the first ones in the class that morning, but we quickly got situated-  about 25 female students all there to learn JavaScript!  I don’t think I’ve seen that many women in one technical class in my life and no matter how much I love hanging out with the guys that I do in the Oracle realm, this was a refreshing change.  The room was filled with women of all ages, all walks of life and no, I was not the only woman in the room with brightly colored streaks in her hair, tattoos or multiple piercings.

yes

Now if you haven’t already done so, I recommend joining Meetup and checking out the groups that are in your area of interest.  I run three groups, (RMOUG Women in Technology, Raspberry Pi and STEM and Girl Geek Dinners of Boulder/Denver)  I’m also part of a number of other groups, including the Big Data, Women who Code and Girls Who Develop It Meetup, which this one day class was offered by.  At $80, it was a great opportunity to dig into a new language and gain a strong introduction to a computer language, even if you didn’t have any previous experience.

Through the day, we learned how to build out a main page, test code through the console log, incorporate java script into our pages and best practices of beginning Java Script.

Now there are two things I will share with you that I feel are great tips from this class that are available to everyone.  It’s two sites for opportunities to continue with your web design/JavaScript education and they are:

  1.  CodePen–  This site demonstrates different examples of webcode, broken down between HTML, CSS and JavaScript, (any combination of 1,2 or all 3…) and you can make changes to the code to see how it impacts the outcome of the graphics and framework.  It really puts how these three interact to build out impressive web designs and where you would use one over another.
  2. Exercism–  This site gives you real world coding problems, allows you to code a solution, submit the solution for valuable feedback.  It’s important to use what you learn every day to improve upon it.  This site gives that opportunity to you.

 

Posted in DBA Life, Oracle, WIT Tagged with: ,

March 21st, 2016 by dbakevlar

I appreciate killing two birds with one stone.  I’m all about efficiency and if I can satisfy more than one task with a simple, productive process, then I’m going to do it.  Today, I’m about to:

  1. Show you why you should have a backup copy of previous agent software and how to do this.
  2. Create a documented process to restore previous images of an agent to a target host.
  3. Create the content section for the Collaborate HOL on Gold Images and make it reproducible.
  4. Create a customer demonstration of Gold Agent Image
  5. AND publish a blog post on how to do it all.

igotthis

I have a pristine Enterprise Manager 13c environment that I’m working in.  To “pollute” it with a 12.1.0.5 or earlier agent seems against what anyone would want to do in a real world EM, but there may very well reasons for having to do so:

  1.  A plugin or bug in the EM13c agent requires a previous agent version to be deployed.
  2. A customer wants to see a demo of the EM13c gold agent image and this would require a host being monitored by an older, 12c agent.

Retaining Previous Agent Copies

It would appear to be a simple process.  Let’s say you have the older version of the agent you wish to deploy in your software repository.  You can access the software versions in your software library by clicking on Setup, Extensibility, Self-Update.

extensibl1

Agent Software is the first in our list, so it’s already highlighted, but otherwise, click in the center of the row, where there’s no link and then click on Actions and Open to access the details on what Agent Software you have downloaded to your Software Library.

If you scroll down, considering all the versions of agent there are available, you can see that the 12.1.0.5 agent for Linux is already in the software library.  If we try to deploy it from Cloud Control, we notice that no version is offered, only platform, which means the latest, 13.1.0.0.0 will be deployed, but what if we want to deploy an earlier one?

Silent Deploy of an Agent

The Enterprise Manager Command Line Interface, (EMCLI) offers us a lot more control over what we can request, so let’s try to use the agent from the command line.

Log into the CLI from the OMS host, (or another host with EMCLI installed.)

[oracle@em12 bin]$ ./emcli login -username=sysman
Enter password :
Login successful

First get the information about the agents that are stored in the software library:

[oracle@em12 bin]$ ./emcli get_supportedplatforms
Error: The command name "get_supportedplatforms" is not a recognized command.
Run the "help" command for a list of recognized commands.
You may also need to run the "sync" command to synchronize with the current OMS.
[oracle@em12 bin]$ ./emcli get_supported_platforms
-----------------------------------------------
Version = 12.1.0.5.0
 Platform = Linux x86-64
-----------------------------------------------
Version = 13.1.0.0.0
 Platform = Linux x86-64
-----------------------------------------------
Platforms list displayed successfully.

I already have the 13.1.0.0.0 version.  I want to export the 12.1.0.5.0 to a zip file to be deployed elsewhere:

[oracle@em12 bin]$ ./emcli get_agentimage -destination=/home/oracle/125 -platform="Platform = Linux x86-64" -version=12.1.0.5.0
ERROR:You cannot retrieve an agent image lower than 13.1.0.0.0. Only retrieving an agent image of 13.1.0.0.0 or higher is supported by this command.

OK, so much for that idea!

So what have we learned here?  Use this process to “export” a copy of your previous version of the agent software BEFORE upgrading Enterprise Manager to a new version.

Now, lucky for me, I have multiple EM environments and had an EM 12.1.0.5 to export the agent software from using the steps that I outlined above.  I’ve SCP’d it over to the EM13c to use to deploy and will retain that copy for future endeavors, but remember, we just took care of task number one on our list.

  1.  Show you why you should have a backup copy of previous agent software and how to do this.

Silent Deploy of Previous Agent Software

If we look in our folder, we can see our zip file:

[oracle@osclxc ~]$ ls
 12.1.0.5.0_AgentCore_226.zip 
p20299023_121020_Linux-x86-64.zip
 20299023 p6880880_121010_Linux-x86-64.zip

I’ve already copied it over to the folder I’ll deploy from:

scp 12.1.0.5.0_AgentCore_226.zip oracle@host3.oracle.com:/home/oracle/.

Now I need to upzip it and update the entries in the response file, (agent.rsp)

OMS_HOST=OMShostname.oracle.com
 EM_UPLOAD_PORT=4890 <--get this from running emctl status oms -details
 AGENT_REGISTRATION_PASSWORD=<password> You can set a new one in the EMCC if you don't know this information.
 AGENT_INSTANCE_HOME=/u01/app/oracle/product/agent12c
 AGENT_PORT=3872
 b_startAgent=true
 ORACLE_HOSTNAME=host.oracle.com
 s_agentHomeName=<display name for target>

Now run the shell script, including the argument to ignore the version prerequisite, along with our response file:

$./agentDeploy.sh -ignorePrereqs AGENT_BASE_DIR=/u01/app/oracle/product RESPONSE_FILE=/home/oracle/agent.rsp

The script should deploy the agent successfully, which will result in the end output from the run:

Agent Configuration completed successfully
The following configuration scripts need to be executed as the "root" user.
#!/bin/sh
#Root script to run
 /u01/app/oracle/core/12.1.0.5.0/root.sh
To execute the configuration scripts:
1. Open a terminal window
2. Log in as "root"
3. Run the scripts
Agent Deployment Successful.

Check that an upload is possible and check the status:

[oracle@fs3 bin]$ ./emctl status agent
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
Agent Version : 12.1.0.5.0
OMS Version : 13.1.0.0.0
Protocol Version : 12.1.0.1.0
Agent Home : /u01/app/oracle/product/agent12c
Agent Log Directory : /u01/app/oracle/product/agent12c/sysman/log
Agent Binaries : /u01/app/oracle/product/core/12.1.0.5.0
Agent Process ID : 2698
Parent Process ID : 2630

You should see your host in your EM13c environment now.

fs31

OK, that takes care of Number two task:

2.  Create a documented process to restore previous images of an agent to a target host.

Using a Gold Agent Image

From here, we can then demonstrate the EM13c Gold Agent Image effectively.  Click on Setup, Manage Cloud Control, Gold Agent Image:

Now I’ve already created a Gold Agent Image in this post.  Now it’s time to Manage subscriptions, which you can see a link at the center of the page, to the right side.  Click on it and then we need to subscribe hosts by clicking on “Subscribe” and add it to the list, (by using the shift or ctrl key, you can choose more than one at a time.

gai1

As you can see, I’ve added all my agents to the Gold Image Agent as subscriptions and now it will go through and check the version and add it to be managed by the Gold Agent Image.  This includes my new host on the 12.1.0.5.0 agent.  Keep in mind that a blackout is part of this process for each of these agents for them to be added, so be aware of this step as you refresh and monitor the additions.

Once the added host(s) update to show that they’re now available for update, click on the agent you wish to update, (you can choose even one that’s already on the current version…) and click on Update, Current Version.  This will use your Current version gold image that its subscribed to and deploy it via an EM job-

agent_upd

The job will run for a period of time as it checks everything out, deploys the software and updates the agent, including a blackout so as not to alarm everyone as you work on this task. Once complete, the agent will be upgraded to the same release as your gold agent image you created!

gaig

Well, with that step, I believe I’ve taken care of the next three items on my list!  If you’d like to know more about Gold Agent Images, outside of the scenic route I took you on today, check out the Oracle documentation.

Posted in EM13c, Oracle Tagged with: , ,

March 15th, 2016 by dbakevlar

As we migrate to the cloud, secured credentials are starting to become a standard in most DBAs worlds.  If you don’t take the time to understand ssh keys and secured credentials, well, you’re not going to get very far in the cloud.

data_denied

Now ssh keys are a way to authenticate to a host without the need for a password.  Its far more secure, but takes a little time to set up and comprehension between public and private keys, along with the impact of pass phrases.

Private keys shouldn’t be shared, as anyone with access to the private key can generate and get access to your host.  The Public host has a generated key that can be used to authenticate the user to another host when the key is implemented to that secondary host.

As RSA generation is the most common and what we will require here for EM13c and OPC setup, we’ll focus our post on this type of ssh key.

Generating an SSH Key

As the user that needs to authenticate to the secondary host or cloud provider, you’ll need to generate the key.  This is a simple process on Unix and Linux and with Windows, Putty provides a great Putty Key Generator program that can be used to perform the task.

In a putty console, again, as the OS User that will need to authenticate, run the following command:

$ssh-keygen -b 2048 -t rsa

The command calls the ssh key generator, asks for the key to be in 2048 bytes, (-b) and the type, (-t) is RSA.

The command will ask for a directory to create the files, but a default .ssh directory will be used from the directory the command was issued from if none is specified.

Three files are generated:

-rw-------+ 1 oracle dba 1675 Mar 11 15:01 id_rsa
-rw-r--r--+ 1 oracle dba 398 Mar 11 15:01 id_rsa.pub
-rw-r--r--+ 1 oracle dba 1989 Mar 14 14:42 known_hosts

Note the read write permissions on these files and protect them as you see fit, especially the private key, (the one without an extension… :))

Now I’ve discussed how to create a named credential in my previous post, but I want to discuss how to use these keys to make sure that they’re set up correctly to log into the OPC and other hosts.

You can validate the ssh key by performing the following:

ssh -i ./id_rsa remote_OSUser@remote_hostname

If you can’t log into the remote server, your named credentials creation has an issue and you need to check for typos or incorrect key usage.

Using an SSH Key to Connect to a Remote Host

Once you create a Named Credential, the key is then added to that remote host so that no password is required to authenticate.

On that remote host, you’ll now see the .ssh folder just as you have one on your local host:

-rw------- 1 opc opc 221 Mar 15 12:46 .bash_history
-rw-r--r-- 1 opc opc 18 Oct 30 2014 .bash_logout
-rw-r--r-- 1 opc opc 176 Oct 30 2014 .bash_profile
-rw-r--r-- 1 opc opc 124 Oct 30 2014 .bashrc
-rw-r--r-- 1 opc opc 500 Oct 10 2013 .emacs
drwx------ 2 opc oinstall 4096 Mar 14 17:43 .ssh

In this folder, there is only one file called “authorized_key” until you create ssh keys on the remote host and the authorized_key file has all the public keys and other ssh keys that are required to authenticate.  You can also view this file and you should see the entry for the local host you created a ssh key and named credential for earlier:

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAy+LQteSnuVmxquHXrA0Vfk6Vi8I4aW/vAgmm1BNpMoIe...........== oracle@orig_hostname

This entry will match the .pub file entry on the hostname for the OS User that you generated and used for your Named Credential earlier.  This is all that’s done when a named credential with an SSH key is used for a target-  It simply adds this information to the authorized_key file.

I hope this short post about RSA SSH keys is beneficial in adding to your knowledge base on EM13c, cloud based authentication and other remote host authentication challenges that may require a little more explanation and until next post, keep curious.

 

Posted in Oracle

March 14th, 2016 by dbakevlar

This is going to be a multi-post series, (I have so many of those going, you’d hope I’d finish one vs. going onto another one and coming back to others, but that’s just how I roll…:))

As I now have access to the Oracle Public Cloud, (OPC) I’m going to start by building out some connectivity to one of my on premise Enterprise Manager 13c environments.  I had some difficulty getting this done, which may sounds strange for someone who’s done projects with EM12c and DBaaS.

fa65f436a4c535257341d334bb62c0b2

Its not THAT hard to do, it’s just locating the proper steps when there are SO many different groups talking about Database as a Service and Hybrid Cloud from Oracle.  In this post, we’re talking the best and greatest one-  Enterprise Manager 13c’s Database as a Service.

Generate Public and Private Keys

This is required for authentication in our cloud environment, so on our Oracle Management Service, (OMS) environment, let’s create our SSH keys as our Oracle user, (or the owner of the OMS installation):

ssh-keygen -b 2048 -t rsa

Choose where you would like to store the support files and choose not to use a passphrase.

Global Named Credential for the Cloud

We’ll then use the ssh key as part of our new named credential that will be configured with our cloud targets.

Click on Setup, Security and then Named Credentials.  Click on Create under the Named Credentials section and then proceed to follow along with these requirements for the SSH secured credential:

opc_em5

Now most instructions will tell you that you need to “Choose File” to load your SSH Private and Public Keys into the Credential properties, but you can choose to open the file and just copy and paste the information into the sections.  It works the same way.  Ensure you choose “Global” for the Scope, as we don’t have a target to assign this to yet.

Once you’ve entered this information in, click on Save, as you won’t be able to test it.  I will tell you, if you don’t paste in ALL of the information from each of the the public and private key file in the properties section, it has checks for the headers and footers that will cause it to send an error, (you can see the “****BEGIN RSA PRIVATE KEY****” and “ssh-rsa” in the ones I pasted into mine.)

Create a Hybrid Cloud Agent

Any existing agent can be used for this step and will then serve two purposes.  It will be both the local host agent, as well as an agent for the cloud, which is why its referred to as a hybrid agent.

We’ll be using EM CLI, (the command line tool for EM) to perform this step.  I’m going to use the OMS’ agent, but I’d commonly recommend using another hosts and create a few to ensure higher availability.

 $ ./emcli login -username=sysman
 Enter password :

Login successful
 $ ./emcli register_hybridgateway_agent -hybridgateway_agent_list='agentname.oracle.com:1830'
 Successfully registered list of agents as hybridgateways.

Make sure to restart the agent after you’re performed this step.  Deployments to the cloud can fail if you haven’t cycled the agent you’ve converted to a hybrid gateway before performing a deployment.

Create Database Services in OPC

Once that’s done, you’ll need to create some services to manage in your OPC, so create a database service to begin.  I have three to test out with my EM13c on premise environment that we’re going to deploy a hybrid agent to.

agent_dep4

Now that we have a couple database services createed, then I’ll need to add the information regarding each new target to the /etc/hosts file on the on premise Enterprise Manager host.

Adding the DNS Information

You can capture this information from your OPC cloud console by clicking the left upper menu, Oracle Compute Cloud Service.

For each service you add, the Oracle Compute Cloud Service provides the information for the DNS entry you’ll need to add to your /etc/hosts file, along with public IP addresses and other pertinent information.

opc_em4

Once you’ve gathered this, then as a user with SUDO privs on your OMS box, add these entries to your hosts file:

$sudo vi /etc/hosts
# ###################################### #
127.0.0.1 localhost.localdomain loghost localhost
IP Address   Host Name    Short Name
So on, and so forth....

Save the changes to the file and that’s all that’s required, otherwise you’ll have to use the IP Addresses for these environments to connect.

Now, let’s use our hybrid gateway agent and deploy to one or more of our new targets on the Oracle Public Cloud.

Manual Target Additions

We’ll add a target manually from the Setup menu, and choose to add a host target:

agent_dep1

We’ll fill out the standard information of agent installation directory, run sudo command, but we’ll also choose to use our cloud credentials we created earlier and then we need to check the box for Optional Details and check mark that we’re going to configure a Hybrid Cloud Agent.  If you’re OS user doesn’t have sudo to root, no problem, you’ll just need to run the root.sh script manually to complete the installation.

agent_dep2

Notice that I have a magnifying glass I can click on and choose the agent that I’ve made my hybrid cloud agent.  One of the tricks  for the proxy port is to remove the default and let the installation deploy to the port that it finds is open.  It eliminates the need to guess and the default isn’t always correct.

Click on Next once you’ve filled out these sections and if satisfied, click on Deploy Agent.  Once complete, the deployment to the cloud is complete.

Next post we’ll discuss the management of cloud targets and hybrid management.

 

 

 

 

Posted in DBaaS, EM13c, Oracle Tagged with: , , ,

March 8th, 2016 by dbakevlar

I’ve been working on a test environment consisting of multiple containers in a really cool little setup.  The folks that built it create the grand tours for Oracle and were hoping I’d really kick the tires on it, as its a new setup and I’m known for doing major damage to resource consumption… 🙂  No fear, it didn’t take too long before I ran into an interesting scenario that we’ll call the “Part 2” of my Snap clone posts.

tire

Environment after Kellyn has kicked the tires.

In EM13c, if you run into errors, you need to know how to start to properly troubleshooting and what logs provide the most valuable data.  For a snap or thin clone job, there are some distinct steps you should follow.

The Right Logs in EM13c

The error you receive via the EMCC should direct you first to the OMS management log.  This can be found in the $OMS_BASE/EMGC_OMS1/sysman/log directory.  view the emoms.log first and for the time you issued the clone, there should be some high level information about what happened:


2016-03-01 17:31:04,143 [130::[ACTIVE] ExecuteThread: '16' for queue: 'weblogic.kernel.Default (self-tuning)'] WARN clone.CloneDatabasesModel logp.251 - Error determining whether the database target is enabled for thin provisioning: null

For the example, we can see that our TestMaster is shown that it wasn’t enabled for thin provisioning as part of it’s setup.

If log into EMCC, log into our source database, (BOFA) and then go from Database, Cloning, Clone Management, we can then see that although we had requested this to be a test master database, when I overwhelmed the environment, something went wrong and this full clone hadn’t become a test master for BOFA:

thin_c5

Even though the database that should be the Test Master is visible from the BOFA database Cloning Management page and highlighted, I’m unable to Enable as a Test Master or choose the Remove option.  I could delete it and I’d only be prompted for the credentials needed to perform the process.

delete_db

For this post, we’re going to say that I also was faced with no option to delete the database from the EMCC, too.  Then I’d need to go to the command line interface for EM13c.

EM CLI to the Rescue

As we can’t fix our broken test master view the console, we’ll take care of it with the command line interface, (EM CLI.)

First we need to know information about the database we’re having problems with, so log into the OMR, (Oracle Management Repository, the database behind EM13c)  via SQL*Plus as a user with access to the sysman schema and get the TARGET_GUID for the database in question:

select display_name, target_name, target_guid 
from mgmt_targets where target_name like 'tm1%';
 DISPLAY_NAME
 --------------------------------------------------------------------------------
 TARGET_NAME
 --------------------------------------------------------------------------------
 TARGET_GUID
 --------------------------------
 BOFAtm1_sys
 BOFAtm1_sys
 EF9FC557D210477B439EAC24B0FDA5D9
 
 BOFA_TestMaster-03-01-2016-1
 BOFAtm1
 893EC50F6050B95012EAFA9B7B7EF005

 

Ignore the system entry and focus on the BOFAtm1.  It’s our target that’s having issues from our Clone Management.

We need to create an entry file with the following parameters to be used by our input file argument-

vi /home/oracle/delete_cln.prop
DB_TARGET_GUID=893EC50F6050B95012EAFA9B7B7EF005
HOST_CREDS=HOST-ORACLE:SYSMAN
HOST_NAME=nyc.oracle.com
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1
DBNAME=BOFAtm1
DB_SID=BOFAtm1
DB_TARGET_NAME=BOFAtm1

Next, log into the EM CLI as the sysman user, (or if you’ve set up yours with proper EM CLI logins, then use that…)

$ ./emcli login -username=sysman
 Enter password :
Login successful
./emcli delete_database -input_file=data:/home/oracle/delete_cln.prop
Submitting delete database procedure...
2D146F323DB814BAE053027AA8C09DCB
Deployment procedure submitted successfully

Notice the output from the run: “…procedure SUBMITTED successfully”.  This isn’t an instantaneous execution and it will take a short while for the deletion and removal of the datafiles to take place.

There are a ton of EM CLI verbs for creating, managing and automating DBaaS, this is just demonstrating the use of one of them when I ran into an issue due to resource constraints causing a failure on my test environment.  You can find most of them here.

After some investigation of host processes, I noted that the swap was undersized and after resizing, the job completed successfully.

 

Posted in DBaaS, EM13c, Enterprise Manager, Oracle Tagged with: ,

March 4th, 2016 by dbakevlar

I fly out on Sunday for HotSos and am looking forward to giving a joint keynote with Jeff Smith, as well as giving two brand new sessions on Enterprise Manager New Features.  IOUG’s Collaborate is just a month afterwards, so the spring conference chaos is officially under way.

hotsos16

With running the RMOUG conference, Feb. 9th-11th, I think you can imagine what my response was like when I realized how much content I had to produce for HotSos’ two sessions and then another four for Collaborate, plus a Hands on Lab.

babyscared

As focused as I’ve been on day job demands for a new product, Oracle Management Cloud, which I’m sure you’ve heard of as it goes through trials, I found myself furiously building out everything I needed for my Enterprise Manager 13c environment.  At the same time, we needed to build and test out the HOL container environment and then Brian Spendolini was kind enough to give me access to the Oracle Public Cloud to test out the new Database as a Service with Hybrid Cloud offering.

I know all of it is going to be awesome, but my brain works like a McDonalds with 256 open drive thrus, so until it comes together at the end, I’m sure it looks pretty chaotic.

mcdonalds

With that said, everything is starting to come together, first with HotSos and then with Collaborate, really well.

HotSos Symposium 2016

This will be my fourth year presenting at HotSos Symposium and where other conferences may have mixed content, this is all about performance.  It’s my favorite topic and I really get to discuss the features that I love-  AWR, ASH, EM Metrics, SQL Monitor, AWR Warehouse.  It’s all technical, all the time and I really enjoy the personal feel of the conference that the HotSos group put into it, as well as the quality of the attendees that are there with such a focused objective on what they want to learn.

That Jeff and I are going to do our keynote on Social Media at HotSos really demonstrates the importance of it’s value to a techie career.  Social Media is assumed to be natural to those that are technically fluent and to be honest, it can be a very foreign concept.  Hopefully those in attendance will gain value in professional branding and how it can further their career.

IOUG Collaborate 2016

Collaborate is another conference where I enjoy speaking at immensely.  The session attendance is high, allowing you to reach a large user base and the locations often change from year to year, offering you some place new to visit.  The venue this year is in Las Vegas at the Mandalay.  There’s so much to do during the event that its almost impossible for you to go outside or do something outside the hotel, ( can you call these monstrosities in Las Vegas just a “hotel”? :))  and I know I only went outside once back in 2014 after arriving.

collab

Joe Diemer did a great job putting together a page to locate all the great Enterprise Manager and Oracle Management Cloud content at Collaborate this year.  Make sure to bookmark this and use those links to fulfill your Collaborate scheduler so you don’t miss out on any of it!  This includes incredible presenters and I know I’ll be using it to try and see sessions for a change!

Along with my four technical sessions, I’ll be doing a great HOL with Courtney Llamas and Werner DeGruyter.  We’re updating last year’s session, (OK, we’re pretty much writing a whole new HOL…) to EM13c and we’re going to cover all the latest and coolest new features, so don’t miss out on this great pre-conference hands on lab!

Hopefully I’ll see you either this next week at HotSos or in April at Collaborate!

 

 

Posted in DBA Life, Oracle Tagged with: , , ,

March 1st, 2016 by dbakevlar

With EM13c, DBaaS has never been easier.  No matter if you’re solution is on-premise, hybrid, (on-premise to the cloud and back) or all cloud, you’ll find that the ability to take on DevOps challenges and ease the demands on you as the DBA is viewed as the source of much of the contention.

too easy

On-Premise Cloning

In EM13c, on- premise clones have been built in by default and easier to manage than they were before.  The one pre-requisite I’ll ask of you is that you set up your database and host preferred credentials for the location you’ll be creating any databases to.  After logging into our the EMCC and going to our Database Home Page, we can choose a database that we’d like to clone.  There are a number of different kinds of clones-

  • Full Clones from RMAN Backups, standby, etc.
  • Thin Clones with or without a test master database
  • CloneDB for DB12c

For this example, we’ll take advantage of a thin clone, so a little setup will be in order, but as you’ll see, it’s so easy, that it’s just crazy not to take advantage of the space savings that can be offered with a thin clone.

What is a Thin Clone?

A thin clone is a virtual copy of a database that in DevOps terms, uses a test master database, a full copy of the source database, as a “conduit” to then create unlimited number of thin clone databases that save up to 90% on storage requirements separate full clone for each would need.

testmaster

One of the cool features of a test master is that you can perform data masking on the test master so that there is no release of sensitive production data to the clones.  You also have the ability to rewind or in other words, let’s say, a tester is doing some high risk testing on an thin clone and gets to a point of no return.  Instead of asking for a new clone, they can simply rewind to a snapshot in time before the issue that caused the problem occurred.  Very cool stuff…. 🙂

Creating a Test Master Database

From our list of databases in cloud control, we can right click on the database that we want to clone and proceed to create a test master database for it:

clone2

The wizard will take us through the proper steps to perform to create the test master properly.  This test master will reside on an on-premise host, so no need for a cloud resource pool.

clone3

As stated earlier, it will pay off if you have your logins set up as preferred credentials.  The wizard will allow you to set those up as “New” credentials, but if there is a failure and they aren’t tested and true, it’s nice to know you already have this out of the way.

Below the Credentials section, you can decide at what point you want to recover from.  It can be at the time the job is deployed or from a point in time.

You have the choice to name your database anything.  I left the default, using the naming convention based off the source, with the addition of tm, for Test Master and the number 1.   If this was a standard database, you might want to make it a RAC or RAC one node.

Then comes the storage.  As this is an on-premise, I chose the same Oracle Home that I’m using for another database on the nyc host and used the same preferred credentials for normal database operations.  You would want to place your test master database on a storage location that would be separate from your production database so as not to create a performance impact.

clone4

The default location for storage of datafiles is offered, but I do have the opportunity to use OFA or ASM for my options.  I can set up Flashback, too.  Whatever listeners are discovered for the host will be offered up and then I can decided on a security model.  Set up the password model that best suits your policies and if you have a larger database to clone, then you may want to up the parallel threads that will be used to create the test master database.  I always caution those that would attempt to max the number out, thinking more means better.  Parallel can be throttled by a number of factors and those should be taken into consideration.  You will find with practice that you find a “sweet spot” for this setting.  In your environment, 8 may be the magic number due to network bandwidth or IO resource limitations.  You may find it can be as high as 32, but do take some time to test out and know your environment.

clone5

Now comes the spfile settings.  You control this and although the defaults spfile for a test master is used here, for a standard clone, you may want to update the settings for a clone to limit the resources allocated for a test or development clone.

Now if you have special scripts that need to be run as part of your old manual process of cloning, you can still add that here.  That includes BEFORE and AFTER the clone.  For the SQL scripts, you need to specify a database user to run the script as, too.

If you started a standard clone and meant to create a test master database, no fear!  You still have the opportunity to change this into a Test Master at this step and you can create a profile to add to your catalog options if you realize that this would be a nice clone process to make repeatable.

clone7

The EM Job that will create the clone is the next step.  You can choose to run it immediately and decide on what kind of notifications you’d like to receive via your EM profile, (remember, the user logged into the EMCC creating this clone is the credentials that will be used for notification….)  You can also choose to perform the clone later.

clone8

The scheduling feature is simple to use, allowing you to choose a date and time that makes the clone job schedule as efficient as possible.

clone9

Next, review the options you’ve chosen and if satisfied, click on Clone.  If not, click on Back and change any options that didn’t meet your expectations.

If you chose to run the job immediately, the progress dashboard will be brought up after clicking Clone.

clone10

Procedure Activity is just another term for an EM Job and you’ll find this job listed in Job Activity.  It’s easier to watch the progress from here and as checkmarks show in the right hand column, the step is completed successfully for your test master or clone.

Once the clone is complete, remember that this new database is not automatically monitored by EM13c unless you’ve set up Automatic Discovery and Automatic Promotion.  If not, you’ll need to manually discover it.  You can do that following this blog post.  Also keep in mind, you need to wait till the clone is finished, so you can set the DBSNMP user status to unlocked/open and ensure the password is secure.

Now that we’ve created our test master database, in the next post, we’ll create a thin clone.

 

Posted in DBaaS, EM13c, Oracle Tagged with: , , ,

February 25th, 2016 by dbakevlar

I thought this was kind of a cool feature- the ability to send a message to appear to specific or all users in the Cloud Control Console.  I have to admit that I used to like a similar feature in Microsoft/MSSQL to send network broadcast messages to desktops that offered one more way to get information to users that they might be less inclined to miss.

em_sysbroad

Anyone who’s already deployed/upgraded to Enterprise Manager 13c and wanted to search how to use this feature, it’s not well documented, so I’m going to blog about it and hopefully that will assist those that would like to put this great little feature to use.

First off, know that the broadcast message is issued by the administrator from the Enterprise Manager command line, (EMCLI.)  There isn’t a current cloud control interface mechanism to perform this.

If you look in the documentation, you’ll most likely search, (like I did) for a verb that has a naming convention of %broadcast%, but came up with nothing.  The reason you can’t find anything is that the verb is wrong in the docs and I’ve submitted a document bug to have this corrected, (thanks to Pete Sharman who had a previous example of the execution, so realized it didn’t match what he had in his examples…)

In the docs, you’ll find the entry for this under the verb: publish_message

The correct verb for this feature is: send_system_broadcast

I ended up pinging Pete because I was concerned that the verb didn’t exist and it took a search for the right key word to find it after dumping out all the EMCLI verbs to a text file. Its a good idea to know how to do this, simply type in the following to gather all the verbs from the library and redirect them to a file that’s easier to parse through with an editor:

$ ./emcli help > emcli.list

You can then view this list and in it, you’ll find the correct verb name:

System Broadcast Verbs
send_system_broadcast — Send a System Broadcast to users logged into the UI

Once you know the verb, then you can request detailed information from the EMCLI verb help command:

$ ./emcli help send_system_broadcast
 emcli send_system_broadcast
 -toOption="ALL|SPECIFIC"
 [-to="comma separated user names"]
 [-messageType="INFO|CONF|WARN|ERROR|FATAL" (default is INFO)]
 -message="message details"
Options:
 -toOption
 Enter the value ALL to send to all users logged into the Enterprise Manager UI enter SPECIFIC to send to a specific EM User
 -to
 Comma separated list of users. This is only used if -toOption
 is SPECIFIC
 -messageType
 Type of System Broadcast, it can be one of following types
 INFO|CONF|WARN|ERROR|FATAL
 -message
 Message that needs to be sent in the System Broadcast. It must have a maximum of 200 characters.

EM CLI verbs can be issued two different ways, as a single command from the host command line interface or internal to the EM CLI utility.  For the command to execute successfully, the login into the EM CLI must be performed, otherwise, you’ll receive an unauthorized error like the one below:

$ ./emcli send_system_broadcast -toOption="ALL" -message="System Maintenance, 6pm"
 Status:Unauthorized 401
$ ./emcli login -username=sysman
 Enter password :

Login successful
$ ./emcli send_system_broadcast -toOption="ALL" -messageType="WARN" -message="System Maintenance, 6pm"
 Successfully requested to send System Broadcast to users.

Note: If you upgraded your EM12c to EM13c, ensure you syncronize your CLI library before attempting using a new verb from the 13c library, too.

I wasn’t as satisfied with the internal CLI utility.  The error messages weren’t as helpful as when it was issued by the command line and then there were odd ones like below:

 emcli>send_system_broadcast (
 ... toOption="ALL"
 ... ,messageType="WARN"
 ... ,message="Applying EM Patch at 6pm MST, 3/1/2016"
 ... )
 com.sun.jersey.api.client.ClientHandlerException: oracle.sysman.emCLI.omsbrowser.OMSBrowserException

So I found that issuing it from the host command line offered much better results:

$ ./emcli send_system_broadcast -toOption="ALL" -message="Testing"
 Successfully requested to send System Broadcast to users.
 
$ ./emcli send_system_broadcast -toOption="ALL" -messageType="WARN" -message="Hello EM Users, Maintenance Outage at 6pm MST"
 Successfully requested to send System Broadcast to users.

The message shows at the top right of the screen and will continue to be displayed until the user clicks on Close-

sysbroadcast2

Now, not that I’m advocating sending bogus or silly messages, but you can have some fun with this feature and send messages to unique users using the specific to option and call to any EMCC user:

$./emcli send_system_broadcast  -toOption="SPECIFIC" -to="KPOTVIN" -messageType="WARN" -message="Get off my EM13c Console, NOW!!"

What does the message look like?

sysbroadcast

And no, you can’t send a specific user message to the SYSMAN user:

Following users are inactive/invalid. Cannot send System Broadcast to them: sysman.

Mean ol’ Enterprise Manager… 🙂

Posted in EM13c, Enterprise Manager, Oracle Tagged with: ,

February 17th, 2016 by dbakevlar

There was a question posted on Oracle-l forum today that should have a blog post for easy lookup for folks.  Regarding your Enterprise Manager repository database, (aka OMR.)   This database has a restricted use license, which means you can use it for the Enterprise Manager repository, but you can’t add partitioning to it or RAC or dataguard features without licensing those features.  You also can’t use the diagnostic and tuning pack features available in Enterprise Manager on the repository database without licensing it outside of the EMDiagnostics tool.  You can view information about the license that is part of the OMR here.

No one wants to be open to an audit or have a surprise when inspecting what management packs they’re using.

horrified

To view what management packs you’re using for any given EMCC page, you can use the console and access it from the Setup menu from EM12c or EM13c:

mgmt_pack

With that said, Hans Forbrich made a very valuable addition to the thread and added how to disable EM management control access in your OMR database-

Run the following to disable it via SQL*Plus as SYSDBA:

ALTER SYSTEM SET CONTROL_MANAGEMENT_PACK_ACCESS='NONE' scope=BOTH;

Other packs are disabled using the EM Cloud Control with the appropriate privileges in the console using the SETUP menu in 12.1.0.4 with a patch or higher:

mgnt_pck_acs

The view can be changed from licensed databases to all databases and then you can go through and adjust management packs as licensed and then apply.

mngt_acs_2

Don’t make yourself open to an audit when Enterprise Manager can make it really easy to manage the management packs you are accessing.

Posted in ASH and AWR, Database, EM13c, Enterprise Manager, Oracle Tagged with: , ,

February 16th, 2016 by dbakevlar

I’ve just returned from another post-RMOUG conference mountain trip and happy to say that another RMOUG Training Days conference has successfully completed.  It’s a massive undertaking that takes an incredible amount of time from our board of directors, volunteers and Team YCC, our conference support company.  In the end, it’s all worth it and no one regrets the loss in personal life or family time to put on the largest Oracle regional user conference in the US.

timrmoug16

As my fourth year as conference director completes, I can look back and see how the conference has changed, how we’ve successfully taken on intriguing challenges and opportunities for improvement.

Highlights This Year

The OWL, sponsored by Oracle Technology Network.  Laura Ramsey and her folks did a great job with the Hands on Labs and events in the huge area designated behind the exhibitor area.  She had a daily schedule that was published in our mobile app and the daily highlights, which kept our attendees abreast of what was going on in the area each day.

Keynote from Carlos Sierra and Mauro Pagano.  These gentlemen have built and supported the tools that provide the deep and detailed answers that many of us in the performance arena live for.  They are also incredible people and to have them share with us what drove them to build the products, the stories behind initiatives and enhancements to where we are today was really enjoyable to listen to.  It was technical, it was personable and we enjoyed sharing it with them.

mandc

Over 100 technical sessions from the best of the best in Oracle, MySQL, OBIEE, APEX, ADF and others!  We really are very lucky here at RMOUG.  We get the best speakers in the Oracle realm-  Oracle ACEs, ACE Directors, Oracle’s technical specialists and new up and coming.  They all want to come to Denver each February to speak to Training Days attendees.

mauro

Lunches with ACEs.  This has been the third year we’ve put this on.  It’s an incredible opportunity to sit down with your Database God and get close up and discuss your technical topic of obsession.  I even got back into the game, sitting in on Ray Smith’s table, (he wasn’t able to attend, my fault, we forgot to take down his table!) and we sat and discussed Enterprise Manager features for an hour.

raysmithace

Women in Technology Round Table.  This went really, really well this year.  I kept the slides to a minimum and the Denver area panel did a great job discussing topics that are at the front of everyone’s minds on how to get women ahead in technology and our children’s education more focused towards tech careers.

Now the Challenge

We are going through the same challenge as any user group is going through in the US. Without the members and attendees who made the conference fantastic this year, RMOUG, a not-for-profit organization, couldn’t put on a conference next year. Here in Denver, many of our membership aren’t aware of the impressive conference they have for the incredible low cost.  Even though the early numbers are showing we had 40% new attendees, (which means my phenomenal team really did succeed in taking on this challenge) we were still down from our goals because many of our older membership didn’t attend this year’s conference.

I network with a number of user groups and user group boards.  I hear the same questions-

How do we find new attendees?

How do we grow our membership?

Our membership appears to be aging out.  How do we attract younger members?

These are the same challenges Training Days is going through and I often have discussions surrounding how to reach new demographics and attendance.  At the same time, its common to have a conversation with someone you do expect to attend that says, “I won’t be at Training Days this year…”

Our previous attendees have become accustomed that the conference will always be there the next year.  The effort that it’s taken to keep this conference active and successful is mind-blowing when I look review the data.  We make this transparent for a reason, but at this time, I think it’s important to know how our investment looks to those outside of Denver and RMOUG.

Our attendees aren’t aware that some of the top database consultant companies around the US are flying in their employees to attend RMOUG Training Days.   Our conference is 1/2 the length of Oracle Open World, KSCOPE and Collaborate, the three larger conferences in the US.  At the same time, for only 1/2 the length, it’s only a 1/5th of the cost and you get 10 tracks and 2 1/2 days of pure technical content.

kosborne

The networking opportunities with the biggest companies in the US, along with your local peers is priceless!

conf16

As for RMOUG’s board and the Training Days committee?  We’ll keep introducing Training Days to new potential attendees and hope that the missing members will return to this impressive and incredible conference.   We’ll keep spreading the word to companies about how inexpensive and valuable the training is at Training Days in hopes the next generation of DBAs, Developers and Database specialists have this incredible opportunity to better themselves and their career.

If we kept this information so transparent, you, as a member or attendee of RMOUG Training Days wasn’t aware of it, start thinking of our Oracle regional user group as our community and that by contributing by attending just has the additional benefit of helping out our own career!  Now I’m going to enjoy a few weeks off before the planning starts again for next year!

Posted in DBA Life, Oracle

January 28th, 2016 by dbakevlar

To assist users as they plan out their upgrades and new Enterprise Manager environments, I wanted to go over a few subtle, but important changes from EM12c, 12.1.0.5 to the newest release, the much anticipated EM13c, 13.1.0.0.

em13c_splash

EM Processes

One of the things you’ll notice when starting an EM12c from the command line is WHAT is started.

$ ./emctl start oms
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Starting Oracle Management Server...
Starting WebTier...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up

The Oracle Management Repository, (aka OMR, the database) has been started separately, along with the listener for the connection to the OMR, but upon issuing the startup command for the OMS, there are steps that are followed in a default installation:

  • OMS, (Oracle Management Service)
  • Weblogic, (Webtier)
  • The Node Manager and a few other necessary components behind the scenes.

You’ll also note that as of 12.1.0.4 with the latest patch and 12.1.0.5, the agent on the OMR host is started automatically:

$ ./emctl start agent
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Agent is already running

Now, with Enterprise Manager 13c, there are a few more processes and checks that are done as part of the start up:

$ ./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Starting Oracle Management Server...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up
Starting BI Publisher Server ...
BI Publisher Server Successfully Started
BI Publisher Server is Up

These two new processes are well known to EM Administrators, but you didn’t see them incorporated into the start up step.

You can see the steps for all the processes started as part of the “emctl start oms” in the $OMS_HOME/gc_inst/em/EMGC_OMS1/sysman/log/emctl.log

2016-01-21 17:26:42,077 [main] INFO commands.BaseCommand logAndPrint.653 - Oracle Management Server is Up
2016-01-21 17:26:42,078 [main] INFO commands.BaseCommand printMessage.413 - statusOMS finished with result: 0
2016-01-21 17:26:42,094 [main] INFO ctrl_extn.EmctlCtrlExtnLoader logp.251 - Extensions found: 1
2016-01-21 17:26:42,095 [main] INFO ctrl_extn.EmctlCtrlExtnLoader logp.251 - Executing callback for extensible_sample
2016-01-21 17:26:42,095 [main] INFO ctrl_extn.EmctlCtrlExtnLoader logp.251 - jar is /u01/app/oracle/13c/plugins/oracle.sysman.emas.oms.plugin_13.1.1.0.0/archives/jvmd/em-engines-emctl.jar; class is oracle.sysman.emctl.jvmd.JVMDEmctlStatusImpl
2016-01-21 17:26:42,200 [main] INFO ctrl_extn.EmctlCtrlExtnLoader logp.251 - rsp is 0 message is JVMD Engine is Up
2016-01-21 17:26:42,200 [main] INFO commands.BaseCommand printMessage.426 - extensible_sample rsp is 0 message is JVMD Engine is Up
2016-01-21 17:26:42,201 [main] INFO commands.BaseCommand logAndPrint.653 - JVMD Engine is Up
2016-01-21 17:26:42,242 [main] INFO commands.BaseCommand logAndPrint.653 - BI Publisher Server is Up

The JVMD, JVM Diagnostics, is now part of the EM infrastructure.  Considering how important java heap knowledge is to tuning your EM environment, it makes complete sense that this is now included with the processing of an EM weblogic host.

There’s a number of new diagnostic reports and health dashboards to assist with ensuring your EM environment retains a healthy performance.

bip13c.

Logs

Along with the important emctl.log, there are some new logs in the sysman logs directory that weren’t there before:

  • emoms_pbs* -Trace and log files for starting worker threads and other background processes.
  • pafLogs  – Not sure, but think this is a sub-directory for plugin logs.  Still researching this one.**
  • jvmdlogs -Directory for JVMD logs
  • syncopss.log -Security log for synchronization with Wallet

**Thank you to Andrew Bulloch: “pafLogs – these are the log files and outputs from the Provisioning and Automation Framework (PAF).  That’s typically DP’s (Deployment Procedures), and other automation tasks that the OMS and, more specifically, the jobs subsystem/tasks subsystem used internally.Resource and Space Usage”

The EM Administrator may want to know, how much user memory is used, upon installation, (not counting heavy activity, plugins, etc.) is used by the added components.

A 12.1.0.5 installation, OMS, OMR with webtier can be tracked with the following when running as a unique user:

ps aux | awk '{arr[$1]+=$4}; END {for (i in arr) {print i,arr[i]}}' | sort -k2

86Mb

For 13.1.0.0, the usage was about 50% higher, (for the basic load, no additional targets or collections occurring):

133Mb

So not much of a difference to start these processes and have them available in the background for reporting and java diagnostic support.

Space required for the OMS_HOME, (sans the logs, GC_INST and the AGENT_HOME) is different as well:

12.1.0.5:  12Gb

13.1.0.0:  14Gb

The Agent has a lot more work to do in the EM13c release and this is why you’ll note I separated the size requirements for releases:

12.1.0.5: 1Gb

13.1.0.0: 3Gb

So there you have it.  A little more background info about EM13c that should assist you in planning for your upcoming upgrade or new environment!

 

Posted in EM13c, Oracle Tagged with: ,

December 21st, 2015 by dbakevlar

I’m thrilled to see the outpouring of support for EM13c already out with it just being announced on Friday!  As technologists all over the world jump in and start installing the newest and greatest version of Enterprise Manager, there is a second group already working on the task of upgrading their existing EM12c environments, (seen a few already, including one from Gokhan!)

13c_cc1

Along with other requirements, like setting optimizer_adaptive_features set to false, there is pre-requisite that I’d like to recommend that everyone perform on any EM12c environment before endeavoring on an upgrade-  do a health check with EM Diagnostics.

Enterprise Manager 12c has been out for four years now.  Although few have had them in production as long as I or a handful of other folks, these environments rarely receive the attention that we give our other production databases and applications.  Due to this, spend a little time ensuring that your environment is ready to upgrade before embarking on the upgrade itself, saving the scramble of work that would be required in the midst of an upgrade and/or a failed upgrade if something was missed.

To perform this, first download and install the latest EM Diagnostic kit using the Oracle Master Note 421053.1.

After installation, simple run a full collection on the environment by running the following command and you will do this from the command line:

$ repvfy diag all

The following diagnostic data will be collected:

  • A “Just the Facts” sweep to tell you if and what problems exist.
  • System reports, including environment, performance and space report
  • More specific sub-system health_report, (crucial as the upgrade will be touching all of this.)
    • loader_health
    • ping_health
    • job_health
  • Performance oriented reports, (what is the current state of the EM health)
    • advisors – awr info
    • user dump
    • backlog report

If you’d like to know more about how to use the EM Diagnostics kit, there’s a great support document, (I’ve named it Werner DeGruyter’s third favorite hobby. You’ll have to ask him about his first two favorites.. :)) 1374945.1.

With that, keep having a great time with the new release and Merry Christmas and Happy Em’ing!

 

 

Posted in EM13c, Oracle Tagged with: ,

December 18th, 2015 by dbakevlar

If you haven’t been watching the latest Oracle news, Enterprise Manager 13c is out!  You can get all the latest information on newest features that were included in this release here and download it from OTN here!

         Merry Christmas everyone!!

christmas-animated-elf

Posted in Oracle

November 10th, 2015 by dbakevlar

Working with test data is always fun and teaches those of us inside Oracle how best to use the new products that will be offered to benefit customers.  To me, it’s pretty straight forward and I didn’t first understand why it was necessary, but as I sit in on calls as we do performance testing of these great, new products, I understand why they need to direct so many, so often.  Anything performance is as much a mindset as it is a science.

Application Performance Monitor, aka APM, is no different.  I didn’t exactly “follow” the hands on labs each time we’ve been taken through them, as I find it’s important for me to run through the product on my own and find out if its intuitive when I use it on my own.

I commonly ask myself a few questions as I review the results afterwards:

  1.  Do I come up with the same conclusions, (or the same page as the lab expected me to,) even if I went about it a different way?
  2. Does the user interface provide me with the pertinent data I need to answer the questions the business is going to be asking when performance issues arise?
  3. How easily might a user be lead astray by data that has nothing to do with the culprit of the performance impact?

apm1109_1

Slow Ajax performance is what we are experiencing in our example today.

Clicking on Ajax Calls will display this information and confirm the customers experience.

apm1109_2

Notice that currently there is almost 40% errors, 257 errors in total. By clicking on Checkout, where the errors are showing, we can then drill down into the details:

apm1109_3

I then click on the link for the procedural call name, (/RideShare/checkout) and it takes me to a page that starts to break down the response time, (including faults, which we’ll focus on today) along with Calls and Errors, plus the Tier Average Response, (which includes the time spent on the AppServer calls vs. Database and External Calls).

apm1109_3

If I click on the Links tab, I can see all the correlated links for this application and how they are performing.  I can quickly see what calls are experiencing errors, (no matter if they are a parent call or a child call):

apm1109_5

And what calls are NOT connected to the issue and aren’t experiencing any errors:

apm1109_6

Although I’m only showing a snippet of the database calls, you can also click on the Database tab and verify that the database is excluded from the issue.

apm1109_7

I click on Instances and it quickly shows a list of everything connected via ECID and reasons for the capture.  For our experience today, we’ve focused on the ones listed by Fault.

apm1109_8

Now I can click on the Reason Captured and I can sort by Faults to see if there is a pattern to what is going on:

apm1109_9

It confirms that all the issues are resulting from the App Server.  If I click on one of the Instances that experienced the faults, it will then display the details of what occurred:

apm1109_10

Notice that the checkout, when it calls the OrderService.submit, there is a failure when the OrderService.submitWithid is called.  If we expand the view for this call, we get to see all the calls involved in this application code and can then see that the error occurs during the RestaurantService.placeOrder.

apm1109_11

If you hover over the time “Self-time”, we find out that over 76% of the total time is spent on this step and is the source of the response time impact.

I can then click on the upper right and leave APM to Log Analytics, (which we’ll cover in detail in future posts) and View Related Logs, which gives me the log data that confirms what I’ve already found out from the Instance Information:

apm1109_12

I prefer to check mark the “Show Message Field” to eliminate the need to parse the message out of the log text, (future tech tip for working with this product…) and again, you see the error that is experienced by the user when they try to submit RestaurantService.placeOrder.

 

Posted in Oracle, Oracle Management Cloud

November 2nd, 2015 by dbakevlar

So I’m about to depart for one day in San Antonio, Tx. to present at the International Performance and Capacity Conference, but I wanted to try to get another blog post out, this time regarding my survival at the 2015 Oracle Open World.

I’m not sure if it just that the schedule was a month later than usual or that it was just more demanding this year, but I left on Friday, feeling utterly exhausted and judging by the tweets and posts from others on social media, I don’t think I was the only one!  I always regret what I didn’t get to do or those I didn’t get to meet, but this year, there were even events that I was supposed to attend that I wasn’t able to!  There were just too many demands on my time-  I’d get up at 5-6am each morning and collapse in bed at 11pm to midnight each night thinking about what I still had to get done!

Demo Time!

Some of my favorite times were at the demo booths. I really enjoyed talking to everyone and getting the chance to introduce the attendees to Enterprise Manager 13c was a joy.  The great new interface and features were impossible not to get excited about!

20151026_112357

I’ve always tried to get everything prepared before I leave for the event and outside of anyone’s control, my poor co-presenter for Oak Table World, Stewart Bryson, had to go in for emergency surgery on his knee.  Rather than do my well-known AWR Warehouse presentation sans the great OBIEE tier from Stewart, I chose to build out something new.  My mistake- I chose to decide on the topic after drinking out with Alex Gorbachev on Friday evening.  I really should know better than this, but instead attempted to build a Raspberry Pi project with missing hardware until I finally had to admit to Alex that I’d coded my way into a hardware corner without the time or resources to get out of.  Needless to say, I found myself without slides or a technical topic on Sunday for my presentation Tuesday morning.

Running Amok

I ended up having to skip the bridge run, (in my arthritis case, walk) and got down to building out a new project after seeing the stuffed bears in the Oracle retail store. I knew I wanted to inspire the attendees to get involved with the younger generation, but what could I quickly build out without too much demands on my time that would do that?  After purchasing one of the bears, I went over to Central Computers and added a PiCamera to my already purchased Raspberry Pi investment.

20151023_153256

Thanks to some jumper wires from Mark Vilrokx and, a few misc. buttons, safety pins and such from stores downtown, I was ready to start operating on the bear.  The amount of odd comments as I sat in the OTN area, cutting open the bear and pulling the stuffing out to insert the Raspberry Pi was pretty humorous.  I had to warn my husband as he went into our room much later in the day, that no, I was not building a bomb and to hide the hardware from the poor maids before they freaked out upon cleaning our room!

20151024_121030

The end product was pretty cool though- The RPI_OracleBear is a picture taking, tweeting  stuffed bear when you pushed the button on his paw!  You can check him out on Twitter, as he was a guest to a couple of the parties and locations throughout the conference week.  The session at OTW went well and I was very happy with the response, especially considering I’d just put it together the last 48 hours between booth duty and other demands.  The worst part was that in the middle of coding the python piece for the project, I lost track of time and missed the Enterprise Manager SIG meeting on Monday evening!  This is where hyper-focusing is more of a liability than a gift.

I was able to make all my other commitments throughout the week, including publisher meetings representing my group, meeting with many customers and getting the word out about Oracle Management Cloud and even demoing Enterprise Manager 13c!  I really noticed just how busy I was when I’d finished having a lunch debriefing with Oracle Education Foundation on Thursday afternoon and the realization hit me that I didn’t have anything to do for the next three hours.  I’d been running non-stop for so many days that I just didn’t know how to handle any downtime, but ended up going back to my room to mail out RPI Oracle Bear, (no one wants to know what would have happened if I’d tried to take him through TSA scanners…) and packed my bags to gladly go home the next morning. All and all, it was a great year, but looking forward to the event next year to be a bit earlier than the end of October! 🙂

oowselfie

Things we learned this year:

  1. Don’t go shopping with Kellyn- You’ll leave with great Doc Martens and Doc will be sending thank you emails to Kellyn to come back anytime and bring more of her friends.  Just ask Danny and Bobby.
  2. Plan out WIT events well in advance- once you get to OOW, it gets to crazy to get plans together.  We’ll get it together next year, really.
  3. Don’t bring Bjorn where he doesn’t want to go.  It’s like taking a three year old to the dentist and he’ll drive my husband right over the edge…:)
  4. Check all meetings in calendar and make sure people have set them to adjust for your timezone.  Nothing like finding out that it’s still on Mountain time and I’ve migrated to Pacific!

Last but not least-  Kudos to Oracle not just for putting on a great conference, but of all things, the backpack attendee gift this year.  I know it sounds strange, but I give away these bags or turn them down.  This year, I talked my husband out of his in less than a day and will be using it on my trips for quite some time.  Its compact, attractive and stays on my small shoulders.  I expect conferences to cater to the majority of attendees and let’s be honest, it’s men, but this bag is something that both men and women found very functional and appealing.  Great job on a small but cool detail for the attendees.

 

 

Posted in Enterprise Manager, Oracle, Oracle Management Cloud Tagged with: ,

  • Facebook
  • Google+
  • LinkedIn
  • Twitter