Subscribe to Blog via Email
How many times have you had a developer come to you and say, “I just did a bad thing in the database. Can you recover from what I just did?”
With Delphix virtualization, we make this pretty easy to address from the user interface with a simple slider to recover from a PIT before the catastrophic mistake, but today, we’ll discuss how to do this from the command line
1.Log into the Delphix engine as an admin user.
ssh delphix_admin@<yourengine> delphix > timeflow delphix timeflow > ls
2. Depending on the platform that you’re using, (in our example, we’ll use Oracle) you’ll see the list of the databases available and can choose the one that you want to refresh before the catastrophic incident from the clueless developer
delphix database> select [VDB name]
3. We can do a simple rollback if we just want to go back to the last snapshot or we can use an list command to see more options:
delphix database "[VDB Name]"> rollback
delphix database "[VDB Name]" rollback *> ls Properties type: OracleRollbackParameters credential: (unset) timeflowPointParameters: type: TimeflowPointSemantic container: (required) location: LATEST_POINT username: (unset)
4. So we’ve decided to do a PIT recovery after the mistake and use the following command and then commit the changes:
delphix database "[VDB Name]" rollback *> set timeflowPointParameters.location=82439
delphix database "[VDB Name" rollback *> commit
That’s all there is to it.
So what if the developer is incompetent and screws up repeatedly?
Follow steps 1-4 above and then to purge the problem from the environment, run the following command from the delphix engine:
delphix database developer [developer employee ID] remove *> commit
An eject button we’ve installed in the delphix engine will remove the developer from the premises and Delphix will even submit all necessary paperwork to Human Resources to complete his termination processing.
If you’d like to automate the process, you can create a handy script that simply asks for the following parameters by calling it from any shell, Powershell for windows or even Jenkins as part of DevOps!
Happy April 1st!
I was surprised on April 20th when I awoke to find a 1.3G OS update on my Samsung Galaxy 6 Edge+. I’d never experienced any issues with an update before, so I quickly connected my phone to the WiFi and let it download then upgrade my phone, anxiously awaiting what new Android features awaited me.
I proceeded through my day, but was concerned as battery usage was higher than usual and I suffered email failures from Gmail and a few tweets didn’t go through. I consider myself quite familiar with mobile phone trouble shooting and promptly performed the standard steps to address issues, but upon the next morning, I was faced with the same issues.
I researched and found that I wasn’t the only one, as numerous Note, Galaxy and even new Galaxy 7 users were reporting similar issues with texts, emails and network connectivity.
I happened to be running work errands and stopped at my neighborhood T-Mobile store to see if they’d heard anything. The tech was surprised by what I’d tried:
Then he was even more impressed with my phone- I have my Samsung set up at the most optimal settings.
I was having an issue clearing the cache partition on the phone and was looking to how it should be done with the 6.01 release. There had been a change in the button combination, (volume down, home button, power button combo brings to a screen instead of clears the partition) and he was able to help me out with this, clearing the partition.
There was a second fix that was added to the clearing of the partition:
The actual 6.01 upgrade system update HADN’T finished! Upon clearing the cache partition, the update completed and many of the issues I was experiencing stopped.
Then the second part of the problem showed itself. To conserve battery, on many Samsung 6 and 6 Edge devices, it was recommended to run in “Power Saving Mode“. In 6.01, there is a change to the features provided as part of this mode.
It now LIMITS the amount of data allowed to be SENT or RECEIVED.
Reason tweets with pictures and emails with attachments failing SOLVED. Take the phone out of “Power Saving Mode” and these emails and tweets stuck in “limbo” should immediately be sent!
So, to summarize- If you are having issues with emails, network connectivity and social media, do two things:
I’m happy I figured this out, as the Samsung Galaxy Edge Plus has been my favorite phone ever, so having it at top functionality after the upgrade was important! Hope these tips to fix issues after the upgrade to 6.01 helps you, too!
Since the introduction of Enterprise Manager 12c, folks have been asking for a list of best practices. I know a lot of you have been waiting for this post!
1.Use previously deployed, older hardware for your Enterprise Manager deployment on 13c.
Enterprise Manager is a simple, single service system. There is no need for adequate resources and ability to scale. In fact, I’ll soon be posting on my blog about building an EM13c on a Raspberry Pi 3.
2. Please feel free to add new schemas, objects and ETL’s to the Oracle Management Repository, (OMR.)
This database doesn’t have enough to do with metric collections, data rollup, plugin, metric extensions and notifications.
3. Turn on the standard statistics jobs and baseline collection jobs on the OMR.
The OMR has its own version of the stats job, but running two jobs should make it run even better and even though baselines aren’t used, why not collect them, just in case?
4. Set the EM13c to autostart, but set the listener to stay down.
The Oracle Management Service, (OMS) shouldn’t require the listener to connect to the OMR when starting, after all.
5. If there is a lot of garbage collection, just add more memory to the java heap.
If we give it more memory, then it will have less to clean up, right? More is better and there isn’t any way to find out what it should be set to anyway.
6. If you want to use the AWR Warehouse, you should use the OMR database for the AWR repository, too.
It shouldn’t make a difference to network traffic, datapump loading or resource workloads if they share a box. These two databases should work flawlessly on the same hardware, not to worry about network traffic, etc.
7. If you have a lot of backlog for job processing on your EM13c, you should trim down the worker threads.
Serializing jobs always speeds up the loading of data.
8. Sizing an Enterprise Manager EM13c is a simple mathematical process, which I’ve displayed below:
(If I didn’t mention it earlier, there will be a quiz at the end of this post…)
9. Never apply patches to the Enterprise Manager tiers or agents.
Each release is pristine and bugs don’t exist. It will only require more work in the way of applying these patches and downtime to your EM13c environment.
10. Patch any host, database or agent monitored by the Enterprise Manager manually.
Patch plans and automation of patching and provisioning is a terrible idea and the only way a DBA can assure if something is done right is if they do it manually themselves. Who needs a good night’s sleep anyway?
I chose to work for Oracle because I wanted to immerse myself in the Oracle Enterprise Manager product. It was the right place for me. It didn’t matter that I had a background as a multi-platform DBA or that I my skill set was deeply involved in optimization of complex, large databases. I knew it was right for me and I trust my instincts. Due to research and the opportunities preesented to me, I have a great career, great peers and wonderful bosses. It was the right choice for me and I ignored a number of people who would have me take their advice on assumptions they had about the situation vs. doing research or asking questions or even a more amazing idea, maybe trust my judgement… 🙂
This also goes for hardware and software. I am continually surprised how often people, instead of asking questions as to what folks requirements are or what challenges they’ve had with previous equipment or software, just make blind recommendations on what they think is cool or what everyone else tells them is cool.
“You have to have a Mac Pro/Air, anything else is just crap!”
“If you install on anything other than HP-UX, well, you’re just an idiot!”
“I won’t even talk to an MSSQL DBA. They are just on Windoze!”
No questions as to what someone is trying to accomplish or respect shown to others to find out what they need, what they know or to really get folks the right answer. This goes for personal technology and hardware. No respect to folks from other technology areas that are professionals in their own right. Some are just teasing, but its surprising how many seriously just feel unless they get what everyone else says are cool, they aren’t cool.
I run into this with my own hardware choices.
This is what’s in my personal tech arsenal right now-
I have had the most ridiculous comments come out of perfectly reasonable and intelligent people’s mouths over all of these products.
“Why wouldn’t you buy a Mac??”
“A Surface Pro 3? If you didn’t buy a Mac Air, you weren’t thinking…”
“I only use Samsung Tablets. If you have a Samsung phone, why wouldn’t you have a Samsung Tablet?” (You can insert the Moto 360 for the Samsung Smartwatch, too… :))
“Why would anyone have a smartwatch? Stupid technology!”
I use my technology every day, constantly and it has to work for me. I research products and test them out for an extensive period of time and if something doesn’t work, I learn from my mistakes and take this into consideration on my next purchase. Even when I sold computers, before entering IT, people would ask me how I made so much money and did so well. Some, [limited individuals] would tease me and say it was because I was a girl on the sales floor, but even back then, I believed you found out the technology people needed and that’s what you sold them. If they were satisfied, they came back and/or sent their friends/family to you.
Here’s why I have what I have:
1. I burn through keyboards. A laptop keyboard lasts me about 6-9 months- SERIOUSLY. I type about 120+ words per minute and I type with a lot of force.
2. If I build a local demo of OEM, DBaaS and an AWR Warehouse, I can do it on 16G, but I really need 32G to have it run adequately. I don’t do this very often, but I do need something that can handle it from time to time.
Due to these two requirements, I have the following:
The ASUS with 32G, (Mac did not have a 32G model at the time) was a great deal at $900. I use it only for work and am using it less these days because the letters are starting to disappear on the keys. Yeah, beating even this very tough keyboard up. I gave another great PC, another i7 to my son after 10 months of use. My son keeps joking about having to use an indellible marker to add the letters back to the keys.
The Surface Pro 3- It has 16G or memory, 256G SSD with an i5 and honestly, it’s a great little unit that when I burn through a keyboard? I just BUY A NEW KEYBOARD! (Nope, can’t do that with a MacAir and if you would like to see what I did to the Asus Zenbook, who’s keyboard is the SAME as the MacAir, I’ll send you a picture of the poor thing… :)) It’s incredibly light, has great bluetooth accessories and travels well, including not having to be removed from my bag at most Airport security queues.
My tablet choice is a Mac iPad Air. It’s a wonderful tablet and I can do most of what I do for work, but there are some aspects that are limited and I can’t fully integrate it with my Oracle tools for work. I wouldn’t give it up for the world, just as I love my Surface Pro 3, I love my Mac iPad Air tablet. They are used for different aspects of my day. If you want to know why I didn’t go Samsung, check out my post on it. I was highly disappointed with the Samsung Pro. The tablet’s OS just couldn’t handle 50% of the work tools that Apple’s OS could.
Now for my Samsung Note 4. I tested out the iPhone 6Plus. Missed my widgets and really rely on them. I don’t like Pebble and find the interface for most smartwatches limited in their use vs. what works well for women. As I’ve said over and over again- where are women’s smartphones 80% of the time? They’re in our purse, so for us, a smartwatch needs to be an extension of the phone with clear notifications. The Moto 360 worked with Android and hands down, it was the best watch for me. I never miss phone calls, always am aware of work emails and important information. I can also exclude any notifications that aren’t critical to my daily life.
So, where this post ends….
I wonder how far ahead we would be in technology if people stopped recommending what worked for them, (except to give reasons it worked for them, which are much more valuable than, “You need to get <insert brand/software/hardware here>!”) and started asking questions and made recommendations off of what fulfilled those recommendations?
How often do those of us who work in optimization and enhancement enter businesses and find the wrong solution implemented because no one asked questions or listened to requirements?
I would love to see some folks stop blindly following what they hear is cool and just do some research and find out the right technology for them. Don’t just do it, do the research, folks and allow others to get what is right for them. If Mac Air is right for you, get it. If a Samsung tablet is perfect for what you are going to do, then get it. If you are drooling over a new Windows smartphone, get it! Talk to a lot of people, read a lot of reviews, test out products, but get what you need. There’s nothing worse than buying the wrong product and then finding out that you can’t do what you needed to accomplish because someone sold you on what was right for them.
Posted in DBA Rants
I am a long time supporter of Samsung products- I’ve always had Samsung phones, (currently a Note 3 owner…) Samsung TVs, monitors and even have a Samsung microwave and refrigerator. I love Samsung products and this has included my tablets, recently deciding to upgrade my Samsung Note 10.1, (original) for a Samsung Note Pro 12.2.
I have nothing against Apple/Mac products. I think they are great, but I’m a long time user of Windows, (it really does what I need…) and I have always loved Android OS.
When it came to the Samsung Note Pro 12.2, I found the size and business apps appealing. The idea that I could have a full size keyboard, which seemed still out of reach with my 10.1 was frustrating for me, as I am a command line girl, so love my typing. The quad-core processor and LDRAMM looked impressive and the reviews were promising.
…So Friday I purchased one on my lunch break and brought it home. I’m a long time Android user, love it and set it up to start restoring, (which has always taken about an hour for the main iterations and a couple hours to restore my main apps…) We are about to switch over from Comcast, as some know, there is a lot of new homes being built out just west of our neighborhood and our connectivity started taking a hit about 5 weeks ago. With the slow recovery, I was sure it was the network, but later into the night and the next morning, I wasn’t able to log into most applications and the restore was only about a 1/3 along. I noted that 60% of the battery use was the screen even though I’d been sleeping. The more I used the tablet, the slower and slower it became to the point of unusable for anything requiring a connection to the internet.
I thought I would verify that everything was coming over correctly from my old tablet and that was when I was served the surprise of my life- my old tablet was putting the new one to shame! My old 10.1 with the dual-core processor was running like a champ and wasn’t impacted in anyway like the new one. I started to do some informal tests between the two and even after running Cleanmaster to remove any cache issues, etc., the old one left my brand, spanking new Samsung tablet 12.2 in the dust.
That was when I started to do some more googling, but this time I added the word, “slow” in the searches and sure enough, there was the info I needed. The 12.2 Pro, the TAB and NOTE versions are having issues with slow charging and slow performance when more than one application is open. For me, having more than three apps open resulted in such poor performance that the tablet was seriously unusable.
I returned it and went down the path that I always wondered when would arrive- Yes, I bought a iPad Air. I am pretty happy with it and the keyboard cover is really awesome. I’ll do another review in a couple weeks, but for those that were considering the 12.2 Pro, even the Note, (the Tab version has slower memory) skip it. Samsung needed more time with this one before it went to market and their choice to release it early has driven me from the product line and into the arms of Apple.
It’s been such a long time since I posted anything on my blog. I had this post, last save time was on Dec. 13th, which is ironic considering the subject and what ended up happening that day. I’m going to ramble a bit, but there is a message here folks… 🙂
I’ve always been a strong believer of the old Hebrew saying, “Fall down seven times, get up eight”. I’ve lived by it, which most who know me well enough, realize that it’s not just part of my DBA methodology, it’s also an integral part of my life. When I was in my 20’s, I wasn’t a techie at all. I was working as an auditor for a company in Colorado when I started experiencing strokes between the ages of 21-26 years of age. I had five documented strokes, that until a very dedicated and brilliant Neurologist diagnosed the cause of, I was told no one could deter from occurring and that I’d be blind before I was 40. Through this doctor’s excellent care and help from a Rhuematologist and Hematologist, they figured out the odd auto-immune issue, combined with a natural tendency to have very low blood pressure so that I was stroke-free for almost 19 years. I’m now onto my way to turning 46, was left with 48% of my visual field gone, (eyes are fine, it’s the striate cortex of the right hemisphere that isn’t able to “translate” data any longer, showing up as damaged in MRI scans.) I have very, very few, long-term memories from about the age of 19-26 years of age. All other damage that occurred was addressed with physical therapy and speech therapy. If you consider the years that I am impacted memory wise, yes, had to start looking for a new career, wasn’t like I could perform my previous one any longer. I started first by selling shoes, then computers, which outside of being a woman in her late 20’s selling computers, which meant I did very, very well, I found out I had a knack for software. That was 1995 and by 1997 I was doing desktop support for a large telecom company who ended up putting me through my Oracle 8 certification classes and made me a DBA by 1999.
Now with the years gone by, children born, (by oldest just turned 18, youngest is 12 yrs old) I thought I was pretty safe and had no worries. I had a couple surgeries and lasik without any issue and was secure in the fact that I understood everything I needed to know about my condition, knew it hindered me from receiving care as much as it helped me to know when I did enter the hospital what to keep doctors from doing to me that might kill me, (no, you can’t thin my blood…no there is no concern about clots, I have a BLEEDING problem, not a clotting one, pretty much get the heck away from me before you kill me…) I have an excellent primary care physician that I appreciate for his knowledge and natural skill in the medical field.
I went in for surgery on November 10th. It was unexpected, but I checked into a hospital that was very aware of my medical history, even went over it with me before they checked me in. As it was unexpected, they had decided to do exploratory surgery due to my abdominal pain and discovered I had a ruptured ovarian cyst and simply removed it all in one swoop to avoid any complications. The standard procedure when many have any type of bleeding, is for the anesthesiologist to lower your blood pressure to lessen bleeding. Many aren’t aware of this- I know I wasn’t. Unfortunately, my old medical condition surprised the anesthesiologist as my blood pressure “dipped” twice, very low and upon recovery, I found I had some new areas that were missing, including “spots” across my central right visual field that were just plain annoying. My vision seemed a bit blurry, too, but when I squinted to clear the vision, which would commonly work for anyone who knows what it’s like to have blurry vision, it just didn’t improve. It should be noted, as with any of my children’s births, previous surgeries, again, didn’t partake in any pain medications. I have a high threshold for pain and found no need of them. I bounced back quickly, but the vision issue and what caused it, was of high concern.
My primary care physician followed up with an eye exam, (always have to prove it’s not the eyes first before you go to the brain…) and then onto an MRI scan once that came back as “eyes all good!” I received a great set of glasses that addressed the distortion and for the first time in my life, the unending glare that happens to many with visual field damage. With the new glasses, I was able once again, to drive at night.
During this follow-up time, I was the lucky recipient on Dec. 13th of kidney stones and had to go into the same hospital ER for these. I had been directed there, even after the surgery, as the best place to go due to a urologist on site. After a visit that shall just go down as the “one of the worst care given at a hospital in my life and lucky I got home alive” situations, I then had to see my doctor for a referral to a urologist to actually get the kidney stone broken up, as my ER trip just ended up in me going in with severe pain and then being released in severe pain, drugged up and in worse shape than when I entered the place. My MRI’s are not back yet and we go in to discuss my procedure to break up the stone with the urology specialist. The nursing assistant comes back to the room and says, “I’m sorry, but I can’t get anyone to perform the procedure on you without a medical release from your doctor due to the assumed stroke from last month.” Which translated to me, returns as “Yes, we know what needs to be done, but due to a mistake that no one in the medical industry will admit to, no one will now perform the procedure you need to have done now unless someone else takes responsibility for any mistakes we might make.”
There are times when you realize that figuring out some medical mysteries are more threatening to your future health than just addressing the results and going forward to get the care you need. My primary physician went after them the next day and we took the whole stroke issue off the table immediately. I ended up going a full week with a kidney stone that required a surgical procedure because so much of the medical profession is too worried about red tape, managing pain vs. addressing the problem and forget about caring for the unique patient in front of them. I will never give up my primary care physician. I have no doubt Dr. Michael Archer will keep me alive for a long time. His “strong-arming” of the medical staff at different locations not only got me what I needed, it also made me realize I needed to request copies of my medical records to see just how poorly I was cared for. If I have to return to an ER or hospital, I’m pretty sure all bets are off. You can be sure I will never return to St. Anthony’s North Hospital and I recommend anyone in the Northern Denver area to avoid it. I am now recovering at home after the surgical procedure being performed on the 23rd of December and have one more follow-up visit to complete the process. The delay in my care put me more behind, caused me more lost time in work- thank God it was the holidays, first time I’ve ever actually take pain medication, yes, kidney stone procedures beat out child birth and other surgeries I’ve had for amount of pain. It resulted in a request for more recovery time- It would have been much more difficult had I not worked for a great company like Enkitec and worked from home. I’ve lost out on almost six weeks of writing time on the EM12c book, playing lead author and helping out with other areas for the book, RMOUG Training Days work that requires my time has gone pretty well, thanks to Tim’s involvement, he’s often been able to answer issues for me when I’ve been laid up.
We, as patients need to be ever vigilant and constantly push for the care we deserve from the medical profession. The damage that can be done because someone didn’t take the time to read a full chart or a nurse didn’t have time to take a full medical history of the patient or understand the patient’s unique physiology, pain thresholds, etc. put us at risk. Removing the humanity from health care is true cause for most of our failures in this area of American healthcare, which we in turn have just responded with more insurance choices to address.
So my final message is this- The medical profession has a few bright stars in it and everyone else is just following what it says in a medical book on an issue or the policies of a hospital. To protect your loved ones when they enter a hospital is an incredibly demanding task. Ask every nurse, every doctor to explain what they are doing and if anything doesn’t function, (the call button, the phone, etc…) demand that they correct the situation immediately. If you aren’t getting the care you need, demand to see the facilities patient rights documentation and file complaints. Everyone deserves to receive the care that will help them live a long and healthy life. If the medical staff can not perform their job, find someone else more qualified to do so.
Posted in DBA Rants
This is my first blog post that isn’t on a technical subject, but this one has been rolling around in my head the last couple days…
Jessica Ridgeway’s home is not far from my own. My ex-husband owned a home just four blocks from hers and my dogs played at the dog park in the open space off of Simms St., two blocks over. I’m 99% sure that Jessica went to the same McDonalds that my youngest son goes to each week before heading over to Boy Scout’s on 100th and Wadsworth and I just shopped at that King Soopers in the same location very recently.
I can’t imagine the pain the family is going through and yes, my children are on a shorter leash, even though they are in their teens. These are all good neighborhoods. There is a golf course just off of 108th in Jessica’s neighborhood and like I said, a really cool open space with a dog park just west of it. North Metropolitan airport is less than a mile on Simms and then you can turn east and head towards the Oracle campus and roads that lead to Flat Irons Mall.
What spurred this blog post? I’ve been obsessed with this story as is the rest of the country and as it’s close to home, maybe even more so. As I’ve read the stories, comments at the bottom of many of the news sites have caught my attention from time to time. We wonder about the monsters that take children and yet we can’t even act with sympathy when leaving comments in news stories.
Some of the less offensive ones, that I can list here:
“The Republicans are to blame..”
“Obama must have taken her…”
“Who would take her, she’s too ugly!”
“I’ve noticed that all parents of abducted kids are obese…”
What is the matter with people? We ask for kindness, sympathy, assistance and for a better world, yet we can’t even have the common decency to act appropriately in a comment section of a news story?
My faith in humanity is again questioned…
Posted in DBA Rants
Yes, still catching up on Blog posts… 🙂
Monday is the official start of Oracle Open World. I planned to be onsite at the Enkitec booth from the time the exhibitors hall opened and I was a few minutes early. I hadn’t received an exhibitors ribbon, (one of the few missing from my extended display from my badge…) and so I waited patiently and chatted with a couple of folks who had attended one of my sessions the day before.
Upon entering the exhibition hall, you realize why vendors flock to this conference in hopes of promoting their company. The south hall of Moscone was a stampede upon the opening of the doors and I have no doubt the other exhibition areas received the same welcome by attendees at those entrances.
I proceeded to spend a few hours chatting with folks at the booth and scanning their badges for an opportunity to receive one of the books we offered in drawings twice a day.
Igloo a break from time to time and headed over to the Oak Table World event. This was a great set of specialized talks put on by the wonderful Oak Table folks, sponsored by Delphix, Miracle and others in the great venue of the Children’s Discovery Museum.
I had the opportunity to catch a took on Monday by Tanel Poder and Doug Burns before sitting down and locking in the keynote speaker for RMOUG Training Days 2013. I had also received a nod from another to announce the keynote for us, as well. During all of these days, I also continued to work on another task as the database track coordinator for KSCOPE, serving ODTUG and filling the seats on that committee. I want to thank those that have signed up to help- Galo Balda, Martin Berger, Randy Johnson, Don Seiler, Kent Graziano and Bobby Curtis.
Tuesday was pretty much a replay- I went to the booth, spent time with the Oak Table sessions, then attended the Tweet Meet. It was a good time and I was able to take some time out and interview with the Oracle Social Media Network group.
We ended the evening out with a few friends at the Stinking Rose, a garlic intensive restaurant. I can honestly say that a good time was had by all and one of the best prime ribs was consumed… 🙂
How many sessions did I attend so far, outside of the Oak Table ones? None, there’s just not enough time to meet with all those that want a bit of your time, spend the time needed at the booth and complete tasks on the side. I have folks asking me for info on 12c database and the only time I’ve spent is investigating EM Express!
Tomorrow is another day- Wednesday, to be exact…
As many know, I’ve taken on the mantle of Training Days Director this year for RMOUG’s Training Days 2013. I don’t have much time in the month of August, but wanted to remind everyone that the submission for abstracts is open and can be found on the main RMOUG website page.
We are also looking for volunteers for the 2013 event. Anyone who is interested, please see the website for details and the volunteer page.
I’m very thrilled to say that I think this will be the best Training Days conference yet! I’ve attended a number of conferences this last year and I can verify for anyone that RMOUG’s is the “best bang for the buck” both in cost and for the amount of quality content we offer! The conference has pretty much extended to a full three days now, as OTN Developer Day has enhanced what was once just our University Sessions on the first day. Training Days is now scheduled for Feb. 11th-13th for the 2013 year, moving us to a Monday-Wednesday line up before the President’s Day holiday weekend.
Registration will be opening in the next week and attendees can already look forward to some nice extras in the registration options. Cary Millsap will offer you the opportunity to sign up for his “Mastering Trace Data” class on the 11th of February. I’ve taken this class in both its extended and one day formats and can tell you it’s well worth the extra cost. We will also offer our excellent University sessions and although I can’t spill much info yet, I’m thrilled with the line-up that is already forming and with the small cost of these sessions, they are worth the chance to get in on these limited seating opportunities!
The 11th is also the much anticipated OTN Developer Day from Oracle. This year Kris Rice and the Oracle Experts have not only created a new line of workshops for folks to participate in, we are going to include more for the DBA by having a workshop by Maria Colgan. She’s one of my favorites and anyone who’s attended one of her presentations should really look to this great workshop offering on the first day.
We are already seeing a number of great abstracts submittals. We expect over 300 this year and will need to trim that down to around 175 presentations in the two days of RMOUG’s main training event. We’ve re-organized the tracks to better suit the changes in the database and development arena, including Big Data, Hadoop and Exadata into more advanced database topics with the “Database Deep Dive” track, along with separating out “Database Tools” from “Application Development” to give them more defined areas to the Database Technologist.
Our MySQL SIG should bring us more presentations on this highly successful group. Their Quarterly Education Workshop sessions are packed, so I expect a high turn out at Training Days, too.
I’ve been thrilled with the feedback, suggestions and comments I’ve received from those who have attended, presented or volunteered. This information has been very valuable in letting us know what to keep, what to change and where we can improve this fantastic conference.
There are a number of web/social media events coming up in preparation for the Training Days conference. Keep an eye out for these by following @RMOUG_ORG on Twitter, using the #RMOUG hashtag, liking us on Facebook and joining the Linkedin Group under Rocky Mountain Oracle User Group
If you have any questions or just curious about how this great conference comes together, please drop me a line at TrainingDaysDir@rmoug.org or at email@example.com.
Thank you again to everyone that have been so supportive to the incredible Rocky Mountain Oracle User Group Training Days Conference!
If you desire a subject that will invoke deep passion and
often combined with disgust from a group of DBA’s, disaster recovery is the
one. It is the subject that rarely we
feel our butts are not out there hanging, no matter how much we’ve attempted to
secure our environment.
I’ve observed a consistent flow of articles, conversation
and email discussions on the subject and it is apparent that rarely is the
business as aware as the technical specialists, (aka the DBA) of just how
vulnerable their environments are. Rarely
are the budget dollars allotted to the task of insuring that systems have the
proper disaster recovery hardware/software in place and/or testing performed in
a regular basis.
It’s easy for the business to see the value in the
production systems. They create revenue
and their value is equal to the dollars they produce. Development and test are more difficult for
them to understand, but most times, they can be justified the first time
production is undermined due to development or testing being performed in one
We now get into backup and recovery. How many backups are impacted by 24X7 shops,
where the only thing viewed by the business is impact to revenue by having to
allocate resources to backing up production and placing it on disk/tape that
offers them no value to that revenue.
Yes, the DBA’s and technical management argue, “All it takes is one loss
of production and you will be thankful that we have that backup…” Until that day comes, many business’ rely on
the robust nature of Oracle, the hardware it resides on and the technical
expertise of the folks they’ve hired to keep it running and never having to
rely on those backups. DBA’s commonly fight on a regular basis for time
to allocate to testing, hardware to test recoveries to and explaining to the
business why it’s important. The
business again looks on this as time that could be better allocated to creating
faster systems to create more revenue and again, impact to what the business is
there for- creating revenue.
The next level is then disaster recovery. All DBA’s know this is the final
gauntlet. We pray to the DBA Gods hoping
for a technical manager with the gift to motivate, sell and help the business
to understand why having standby’s of production databases to keep revenue
flowing in case of primary production going down is important. We are willing to sacrifice small animals in
the name of a secondary data center for disaster recovery testing. We want to know how long the business will be
down in case the unthinkable does happen and if all the documentation on what
it takes to create production will actually work when we do try to recreate it. We would also like to have that answer
demanded of us when the unthinkable does happen and upper management is sitting
in front of us asking, “So HOW long are we down for???”
This is not rare, this is not uncommon, it is all too often
the norm for most DBA’s in the business world.
IT Managers, Network Administrators, Database Administrators often
battle day in and day out, not just for what they need to provide the growing
demands of the business, but what the business needs to survive in case of
disaster. Our jobs are not just to
provide you with production, but to provide you with an ability to sustain your
business when the unthinkable happens.
Posted in DBA Rants
A blog post by Simon Cooper sent up a reminder of a subject that has been at the forefront of my mind for the last couple weeks and is a follow up to my blog post The Superman Conundrum. I’d had lunch with a previous coworker from years ago about less than stellar management behavior directed towards him as an employee and also spoke last week to another previous co-worker about challenges in her current workplace. I’m pretty content with how I am treated at my company. Even though the company does not comprise of techies, they at least attempt to understand the demands of the IT industry, giving folks off time during the day when appointments and personal issues arise, knowing that we often “pay-it-forward” within a week or so in after-hours work that is common for most operations support roles.
Simon’s post discusses what should be common sense in a business- Give employees what they need to be as productive as possible, pick your battles and you will reap the benefit in return for the business. You want loyal employees- ones that come to work, look forward to coming to work, drive for new challenges with gusto and always give their best to the company that hired them. Unfortunately, their morale and loyalty is often tested by management and management practices that have no common sense behind them.
The simple policy of treating your employees well and in return you will receive their best is a rarity in the work place. Having this policy, with the understanding that if abused, then there was a mistake made in hiring them and then the company has the opportunity to replace the individual should be in place everywhere when it comes to Information Technology. Instead we commonly hear of companies mistreating skilled professionals and in return, spend ridiculous amounts of time repeatedly in the recruiting phase due to consistent turnover which is a huge waste of any company’s time and money.
The employees that a company such as this will retain are either:
1. possessed of little self-esteem, which in turn shows itself in lack of initiative and personal growth, (doesn’t serve any business well in the long run.)
2. Aren’t shining examples of good employees anyway and are perfectly happy mistreating the company as they are mistreated, (also an additional burden to those that have stuck around with the low self-esteems…)
I think most techies will agree with me- Information Technology is a peculiar industry. Many of the folks that fall into the field have a broad range of skills, often quite intelligent and unique personality traits vs. other areas of business. Since most companies prefer to promote from within the department that the group will be managed from, this results in techies being turned into managers. Where the problem of management promotion being based on control issues more often than true leadership skills, this is never more apparent than in IT. Sorry folks, it’s just a major fact as it is a problem in business overall. We say, “Our Manager is being unfair and effecting morale!” and someone is going to undeniably say, “It’s a job, you’re not supposed to like it, man up, (or woman up, depending on the case)!”
The deal is, most companies should be paying more attention to these types of complaints. Often employees aren’t too keen to complain in fear of reprisal to begin with, let alone stating what the REAL problem is. Keeping an employee happy and productive is actually quite easy and involves very little effort than simple respect. It baffles the mind how often it is more important to a manager to “be right” than to do the right thing for the long haul.
Take the following scenarios:
Jennifer has worked a year for her company and has performed well in her manager’s eyes. He relies on her often for last minute jobs and she has always come through. She suddenly needs the afternoon off , her son is sick at school and has to pick him up. Jennifer’s boss asks her to take sick time to do so and follows up with an email asking her to correct the time in the system because it was short what he feels was a 1/2 hr from what she really took off. Jennifer has only five sick days a year and often performs after-hour support for the company, interrupting her personal life to address issues, losing time with her kids and spouse to address them. There is no “comp time” that she is given for this work- it is just expected from employees at her company.
John has been employed for years by the same company and is in good standing with his current manager. John lost his son in a car accident three months ago. His family has had a very difficult time with the loss and John has a large cache of vacation time that he is pestered by HR to take. When his wife has an extra difficult day due to the loss of their son, he has requested to take a vacation day. John now has an interim manager who refuses to OK the vacation day and chooses to leave John unpaid for those days. John is the sole provider for his family and this is causing even more stress in his personal life. John also performs after hour support and rarely is offered any type of comp time for this time away from his family.
These above scenarios were both opportunities to build employee loyalty but poor management choices, driven by control issues have ended up costing employee loyalty, morale and in the end, company time and money.
The company has taken the time to hire professionals- treat them as you would wish to be treated if you are their managers. Make them realize the value they have to the company as they ARE what comprises the company. I have seen this wonderful pay-it-forward attitude in action when I worked for the Parent Company, formally known as eToys, Inc. They took four months to hire me and once there, I walked around and was impressed to realize the quality of the folks in every department. I give high kudos to what a great manager Steve Ridley, the director of technical operations, along with the Manager of the DataWarehouse group, Greg Sitzman and our CTO, Chris Cummings all were. Steve, as my direct manager, always challenged me, treated me with respect and if I had an emergency come up or needed an afternoon off, he simply stated that my family must come first and he meant it. I came in everyday wanting to work there, do my best and you better believe I put in any after hours support they needed from me. I made sure I was always available and had a sense of pride for the company that I belonged to. This was a direct result of feeling valued and important to the company for the part I played. They understood that employee loyalty was earned by treating everyone with respect, no matter what part you played on the company ladder.
“Jennifer” and “John’s” scenarios above could easily have been turned into a win for everyone. If “Jennifer’s” manager had supported her situation, stating, “I always appreciate everything you do for us and the after-hours support you put in, please, go take care of your family and I will see you tomorrow….” how much time just in emails, time entry and frustration could have been saved to the company? If “John’s” interim manager, no matter what his personal management style was, had put himself in “John’s” shoes for just a moment and said, “John, I know you’ve had a rough time the past couple months and I know you already have an agreement in place with your previous manager. Please, go tend to your family’s needs…” how much could he have earned John’s loyalty if he was interested in making that interim position permanent and in making John a more productive employee the next day when he was back after taking care of his family?
I hear time and time again from different companies and even through recruiters how exhausted certain companies are of finding good people, but I also hear from impressive candidates looking for employment stating that they won’t even phone interview with certain companies due to this type of behavior. Before any company starts complaining about how lacking the hiring pool is, it might be a good idea to look internally and find out if you are losing perfectly good people due to the lack of care and feeding that should be common sense so that a company is able to retain those valuable resources once originally hired.
Posted in DBA Rants
I used to collect comic books when I was a kid, so superheroes are close and dear to my heart, but just not as crazy about the expectation that technical support experts be held to the expecation of infallible like Superman.
I’ve both experienced and have discussed with other technical folks regarding this situation that, as of late and in all common sense, just doesn’t make any sense in the technical world.
I’ve nicknamed this technical environment situation the “Superman Conundrum”. In my history I’ve *survived* three companies who’ve treated their technical environment in this manner. This is the kind of company where hardware, software, applications and processes fail on a consistent basis with the expectation that DBA’s and other support staff will come in and save the day/night/weekends….you get the picture…
The reason for the environment being in such poor condition is commonly due to the following:
For a DBA in this type of environment, it can be both invigorating and terrifying- kind of like being on the high wire without a net. It’s thrilling to come in and save the day, to be able to bring back a system or save a process that everyone thought was doomed. Users and Managers think you walk on water, so it is something akin to an adrenaline high. If it starts to become a repeat performance…often, you then begin to realize, just how foolish and dangerous it is to depend on any human being to be this infallible just because something in the environment should have been automated/built with redundancy/mirrored/designed without such flaws and that’s when you recognize that missing net below you…. Management and peers outside the demands to be Supermen do not see the harm of the missing net, you are Supermen, what’s a little fall? You are infallible, right?
Many of these environments also start to play what I like to call the “blame game”. It first appears that the people involved are attempting to help ensure that processes are put in place to keep the problem from repeating itself. The slight difference is that it results in simply pointing fingers at the “Supermen” they once praised, instead telling them what they did wrong, what they should have done, (in hindsight of course) differently to keep the problem from occurring, (and of course, regularly demands more checks, double checks, etc. to compensate for what flaws are in the environment/software/processes, etc assigned to, you guessed it- Supermen…) or just outright complain without even any suggestions. When it comes to anyone who actually takes the time to suggest an option to correcting the problem at its source, (i.e. fix the code, replace the insufficient hardware, upgrade, tune, etc…) this is quickly and quietly turned down or refused due to time/resource/budget constraints.
There is a backlash condition post the blame game. Little by little, the environment, if on it’s destined course, becomes more and more overwhelming for the “supermen”. This is an exhausting situation, as personal life is intruded upon and professionally it’s not so great to constantly be in fire-fighting mode. In turn, their morale is whittled away at by the blame game until the “supermen” start playing the “why even try anymore game”. You begin to see the looks of hopelessness on the supermen faces. It’s there when they walk in the office, it’s there with the first sarcastic joke of the day and it’s there when you receive email at 2am in the morning in regards to the latest fire they had to use their “super breath” on to put out.
Soon, the turnover starts. The supermen know it’s coming, some may warn you, most won’t say a word and will just quietly leave. The one thing you should know is that the technical world is a small place and even if they don’t let you in on the chaos that exists in your environment, that doesn’t mean they won’t share this information with peers in their field when asked. This can be detrimental to companies when they are looking for new “supermen” and can’t figure out why they can’t get any solid resumes. I have a “list” of companies that I refuse to interview with. Trusted sources have listed them as poor work environments and I have surprised more than one technical recruiter when I’ve been aware of an odd quirk or two in a company that they later on verified. Being a DBA is tough enough, so if we can avoid a poorly managed environment, we’ll commonly do so!
So as a manager, what can you do to try to quell the “Superman Conundrum”?
If you’re a DBA in this type of environment, one who, like Superman, thinks you are not just going to save the day, but going to stabilize and productionize it, I can tell you from going through this myself, it’s not a whole lot of fun. You often feel like you’ve taken two steps forward only to be thrusted 2 1/2 steps back. You may implement automation, request meetings, document processes and standards. The problem always lies in that this is not where management’s focus is. They are sure, that if you were just more perfect, more like Superman, then the environment would run just peachy. It will always come down to either you or the database that is at fault and you must either change that culture of thinking or hope you have a manager around who can help fight this battle for you so you can simply do your job.
It takes a lot of power to take on this kind of culture fight, but heck, you’re Superman, right? 🙂
As busy as I am these days, I am seeing a light at the end of the tunnel, (no, it’s not a train!) We have a new DBA training that is doing bang up job and I do believe there is some lull in the demands of our busy season. As a database administrator, I’m happiest when I have a number of demanding tasks, along with mysteries to challenge me that often, the business isn’t even aware of the level of importance it is to having resolved until I’ve implemented the resolution and they have reaped the benefits.
I have two solid managers that allow me a wide berth to allocate many of my own tasks, along with the ones that they designated high priority. I have quite a wide bandwidth and my ability to multi-task at a dizzying degree allows me to focus on more tasks than most people are comfortable with. I’m a very low-maintenance employee, so it is rare for them to have to come smack me with something hard for not working on what I should be, (although when I’m telling one of them what I think he needs to hear than what I think he wants to hear, I’m sure he would like to hit me up-side the head just for the fun of it…:))
One of these managers is gracious enough to call me his “pinch hitter in the ninth inning of the world series”. As for many business’ that world series is now- we are exceptionally busy, have been building to this time in the year when demands run high and systems have to perform top-notch. This requires all of my energy, time (and often brain cells) to ensuring I’m in top-notch, gung-ho, fire-fighting mode. The demand to place my project DBA skills on the back burner during these crucial service times where quick-thinking, ability to assess situations at blinding speed and low impact, high resolution skills are essential.
As rewarding as this is, (and shows you just what you are made of!), I’m looking forward to the opportunities ahead to bring my project skills back to a fore-front. A quality technical project, well done is something to take great pride in. Once the season slows a bit, I get to work with new technology, test and implement new projects that will offer our business more opportunities to grow that we identified during the last number of months. To return to new feature and design projects that are put on hold to ensure that priorities are met for the business at it’s high time, is a nice change of pace and widens the area of interest for any technical resource.
I am about to take on a number of projects involving 11g features with ASM, OEMGC, data guard, Apex, partitioning and parallel execution. I have another set of project work that involves SQL Server reporting services new features, including work that bridges to our Oracle environment and work more with our MySQL environments. This is when a DBA gets to stretch their technical legs and learn new features and gain knowledge that is put on hold while supporting the current production environment.
While working on these projects, research will need to be performed to ensure we are making the best decisions every step of the way. This then allows another aspect of education that we rarely get when we are in fire-fighting mode- learning in a slower, more controlled environment vs. hurried and very specific knowledge to solve a specific problem that is creating the “fire”. Each of these decisions will be carefully discussed with the DBA team and documented. No hurried trouble-shooting document added to handbooks or emails sent with “up to the minute required” information. These are quality documents that contain all of the data that is needed to support the project work being performed, including the goal, scope and requirements for the project, (and should for any company’s project!), not just the technical details behind the project.
As the adrenaline rush ceases and demands/hours come back more in line with where we all like to be, there is a sense of satisfaction knowing that the business needs were satisfied and that we were able to deliver what people needed to be successful in their own positions to offer the company more success in turn.
As a DBA, you can’t ask for a whole lot more… 🙂
This question seems to pop into my mind consistently over the years as a DBA. I’m a “build it right or don’t build it at all” kind of DBA, but due to my gift for finding problems and fixing them, I find myself more and more often performing the second build on processes/procedures/designs, which I often would like to avoid. I went through this repeatedly at a previous shop and it’s still fresh in my mind, even today…
Don’t get me wrong- I think it’s a noble cause when you first come into a new shop and it’s either been neglected or didn’t have the DBA support it may have needed and the update in TLC is always appreciated, but if it’s something I’m revisiting because of a lack of requirements or a production change that wasn’t thoroughly benchmark tested, I find me getting a little cranky- (usually with myself since I expect that I should be able to read minds at this point! :))
I believe to utilize your DBA and development team efficiently, not wasting money on resources, building “right” is essential. Designing the initial database, as well as future development should be reviewed repeatedly to verify you can answer the following questions with a resounding “YES!”:
The companies I’ve experienced DBA’s overstressed and over-worked in the past are commonly places who can not answer yes to the questions above. Without these requirements in place, the databases become kin to a house with a foundation made of Popsicle sticks, (often built on a side of a mountain prepped for a future mudslide, BTW…:))
If a DBA is not there for the initial build of a database, as often is the case, and is just the “unlucky home buyer” of the database, then there is a unique opportunity in front of you to attempt to correct and change the culture that has built this rickety structure. Changing a culture is a difficult challenge- I won’t try to pull the wool over any one’s eyes. It is attainable and if you have management support, you will already have a huge head start!
If you do have management’s support, what steps should you take to stabilize an environment that is expecting it’s DBA’s to be infallible instead of it’s databases?
The subject of building/designing robust database environments is one I take very seriously and I may take this subject a bit too much to heart at times. I was raised with a tough, dedicated work ethic- something I received from both my parents- rural Canadians are a pretty tough bunch… Taking on a challenge is right up my alley and I’m just not one to give up. Can I accomplish it? Eh yah, you betcha! 😛
Do you ever get that feeling that something just isn’t right, but you have no experience to go on and you’re just too busy to check up on it? I thought it was a bit strange that there wasn’t more communication between the RMOUG management and speakers, but it’s my first time as a presenter, so I decided I was being too high maintenance and didn’t follow up, just checking the site periodically for anything that may pertain to persenters.
so this morning I noticed that I hadn’t received an email from someone in my Yahoo account and decided maybe I should check and see if it was caught by the spam filter. As I sifted through the emails, I came across two emails from the “RMOUG Speakers” email address and I pretty much stopped breathing. When they contacted me in October to say my abstract for my presentation had been approved, the reason they had contacted me by phone was that I had not received their emails. We quickly changed my email address over from my Yahoo account to another, eliminating the problem, unfortunately, it appears it switched back to the registration email since then.
The two emails were informing me of deadlines to submit my supporting white paper and presentation slides by, (both of which were due this last week…) I contacted them as soon as I finished reading the emails and submitted first, my slides, then attained an extended deadline of Friday for my supporting white paper. I already have 13 pages in a word document that was written as my pre-cursor to the slides, so it’s going to be pretty easy to finish it up. Luckily there wasn’t anything else I was missing and I also made sure I have the exact URL to check for updates regularly since I will not be trusting email any longer!
I still feel like a deadbeat RMOUG presenter. I wonder what kind of punishment goes with that?
Posted in DBA Rants
I’ve found myself “whining” in the middle of an Oracle Freelist email conversations in regards to patching- or the actual need to patch.
As a DBA, I have a love/hate relationship with patching. I love that if there is a bug in a version of Oracle, I can simply log into Metalink, (or My Oracle Support or MOS, what else are we calling it these days? :)) look up the error, verify with the documentation or with Oracle’s support and apply the patch- Yay me!
Why am I hatin’ on the patch? My strong feelings are towards the CPU and PSU patching. Did I get stood up on a blind date with a patch? How could I have such strong feelings against patching? No matter how much the ADHD, compulsive, DBA Mommy in me wants to do what’s best for my databases and I’m being told by the experts that patching regularly is the best thing for my databases, there are almost always “side-effects”. Bugs that emerge from the patching that requires more patching or even worse, complexities that have no patching available currently or planned.
Explaining to management, developers, analysts and developers why new bugs have popped up after you’ve just taken an outage to address one or more bugs with a patch is not fun…
So….as DBA’s, what do we choose? Should patching be like poor quality immunizations, resulting in secondary diseases being introduced, requiring more immunizing? Should the cure be worse than the disease? What is the right balance and when do we justify the risk?
I applied my first PSU patch- yes, that’s right, my FIRST PSU patch. I’ve been a DBA for quite sometime and yet this is my first time applying the patches called PSU or CPU. I’ve been a one-off patch girl for so long, but I finally took some time, put aside my feelings and didn’t create another work-around.
I performed extensive research before I decided the Oct2009 PSU patch was the correct decision and had the right combination of patches to address the issues we have been working around for sometime. The patch did require a one-off patch update from Oracle support that conflicted with the PSU Patch, but other than that, (oh and pretty much writing up the correct set of instructions, since Oracle hasn’t even updated the commands that are required for checking conflicts for patches in the instructions!) it all went pretty smoothly.
One week has gone by since I applied the patch to my first production system. Since we like to pummel Oracle features into the ground at my place of business, I’ve had to apply two work-arounds to address issues that arose post-patch. We are starting to prepare for our 11g environments- so this is probably my first and last PSU Patch for 10g- sorry Oracle…
11g patching….Woohoo, can’t wait… 🙂
OK, Who else out there has a 3.8TB tablespace? Anyone, anyone? Bueller, Bueller??
I know, I know- Oracle can support terabytes and terabytes of data in one tablespace, so what is my complaint? Most developers, both database and application do not have the technology to access that data efficiently enough and DBA’s rarely have the disk read and write hardware that will support the kind of pressure that will result from large tablespaces.
I am moving more and more of my hardest hit data to ram disk, but it’s expensive and all of us know the challenges at budget time when you have the choice of cheap, slow disk vs. expensive, fast disk. Pointy haired bosses are not going to see the difference until they start utilizing the database themselves and then, as we all know, the database is guilty until proven innocent.
To compensate for this, I am a firm believer in smaller, narrow tables in star schemas in smaller, easier to manage tablespace “chunks”. I vow to the DBA Gods that I will prune off partitions, first chance I can get ANYONE to sign off on a retention period and teach my developers the power of materialized views, step-through tables, staging tables and to tread lightly in the database.
Now, to go mark my calendar, most likely for one of the holidays coming up to move 3.8TB to more manageable tablespaces and make the developers fix their code that put it there in the first place! 🙂
Posted in DBA Rants