Subscribe to Blog via Email
Follow me on TwitterMy Tweets
Delphix Engineering and Support are pretty amazing folks. They continue to pursue for solutions, no matter how much time it takes and the complex challenges they’re faced with supporting heterogenous environments, hardware configurations and customer needs.
This post is in support of the effort from our team that resulted in stability to a previously impacted Solaris 11.2 cluster configuration. The research, patching, testing and then resulting certification from Oracle was a massive undertaking from our team and I hope this information serves the community, but in no way is recommended by Delphix. It’s just what was done to resolve the problem, after logical decisions for the use of the system by our team.
$ echo “::memstat” | mdb -k Page Summary Pages MB %Tot ———— —————- —————- —- Kernel 151528 3183 24% Anon 185037 1623 12% ...
We can also look at it a second way, breaking down the kernel memory areas with kmsastat:
::kmsastat cache buf buf buf memory alloc alloc name size in use total in use succeed fail ------------------------- ------ ------ ------ --------- --------- ----- kmem_magazine_1 16 3371 3556 57344 3371 0 kmem_magazine_3 32 16055 16256 524288 16055 0 kmem_magazine_7 64 29166 29210 1884160 29166 0 kmem_magazine_15 128 6711 6741 876544 6711 0 ...
Oracle ZFS ARC Cache
Next- Oracle ZFS has a very smart cache layer, also referred to as ARC (Adaptive replacement cache). Both a blessing and a curse, ARC consumes as much memory that is available, but is supposed to free up memory to other applications if it’s needed. This memory is used to supplement any slow disk I/O. When inspecting our environment, a significant amount was being over-allocated to ARC. This may be due to the newness of Oracle 12.2, but in a cluster, memory starvation can be a common cause of node eviction.
We can inspect the size stats for the ARC in the following file:
This assumes ZFS is mounted on /proc, so your actual arcstats file may reside in a different path location than shown above. Inside the file, review the following information:
cis the target size of the ARC in bytes
c_maxis the maximum size of the ARC in bytes
sizeis the current size of the ARC in bytes
Ours was eating up everything left, taking 100% of memory left, as we’ll discuss in the next section of this post.
Oracle Clusterware Memory
The Oracle clusterware is a third area that was investigated for frivolous memory usage that could be trimmed down. There’s some clear documented steps to investigate issues with misconfigurations and feature issues from Oracle that can assist in identifying many of these.
So, post upgrade and patching, what can you do to trim down memory usage to avoid memory upgrades to support the cluster upgrade?
$ crsctl stop res ora.crf -init
$ crsctl delete res ora.crf -init
$ tfactl uninstall
$ srvctl cvu stop $ srvctl cvu disable
$ export ORACLE_HOME=<Grid Home> $ export ORACLE_SID=<Local ASM SID> $ sqlplus / as sysasm alter system set "_asm_allow_small_memory_target"=true scope=spfile; alter system set memory_target=750m scope=spfile; alter system set memory_max_target=750m scope=spfile;
High CPU usage features can be troubling for most DBAs, but when it’s experienced on development and test databases that are often granted less resources to begin with vs. production, a change can often enhance the stability and longevity of these environments.
alter system set "_disable_highres_ticks"=TRUE scope=spfile;
This is the Part III in a four part series on how to:
In Part II, we finished upgrading the Dsource database, but now we need to get it configured on the Delphix side.
Log into the Delphix console as the Delphix_Admin user and go to the Manage –> Environments.
Click on the Refresh button and let the system recognize the new Oracle Home for DB12c:
Once complete, you should see the 12.1 installation we performed on the Linux Source now listed in the Environments list.
Click on Manage –> Datasets and find the Dsource 11g database and click on it.
Click on the Configuration tab and click on the Upgrade icon, (a small up arrow in the upper right.)
Update to the new Oracle Home that will now be listed in the dropdown and scroll down to save.
Now click on the camera icon to take a snap sync to ensure everything is functioning properly. This should only take a minute to complete.
The DSource is now updated in the Delphix Admin console and we can turn our attentions to the Linux target and our VDBs that source from this host. In Part IV we’ll dig into the other half of the source/target configuration and how I upgraded Delphix environments with a few surprises!
I was surprised on April 20th when I awoke to find a 1.3G OS update on my Samsung Galaxy 6 Edge+. I’d never experienced any issues with an update before, so I quickly connected my phone to the WiFi and let it download then upgrade my phone, anxiously awaiting what new Android features awaited me.
I proceeded through my day, but was concerned as battery usage was higher than usual and I suffered email failures from Gmail and a few tweets didn’t go through. I consider myself quite familiar with mobile phone trouble shooting and promptly performed the standard steps to address issues, but upon the next morning, I was faced with the same issues.
I researched and found that I wasn’t the only one, as numerous Note, Galaxy and even new Galaxy 7 users were reporting similar issues with texts, emails and network connectivity.
I happened to be running work errands and stopped at my neighborhood T-Mobile store to see if they’d heard anything. The tech was surprised by what I’d tried:
Then he was even more impressed with my phone- I have my Samsung set up at the most optimal settings.
I was having an issue clearing the cache partition on the phone and was looking to how it should be done with the 6.01 release. There had been a change in the button combination, (volume down, home button, power button combo brings to a screen instead of clears the partition) and he was able to help me out with this, clearing the partition.
There was a second fix that was added to the clearing of the partition:
The actual 6.01 upgrade system update HADN’T finished! Upon clearing the cache partition, the update completed and many of the issues I was experiencing stopped.
Then the second part of the problem showed itself. To conserve battery, on many Samsung 6 and 6 Edge devices, it was recommended to run in “Power Saving Mode“. In 6.01, there is a change to the features provided as part of this mode.
It now LIMITS the amount of data allowed to be SENT or RECEIVED.
Reason tweets with pictures and emails with attachments failing SOLVED. Take the phone out of “Power Saving Mode” and these emails and tweets stuck in “limbo” should immediately be sent!
So, to summarize- If you are having issues with emails, network connectivity and social media, do two things:
I’m happy I figured this out, as the Samsung Galaxy Edge Plus has been my favorite phone ever, so having it at top functionality after the upgrade was important! Hope these tips to fix issues after the upgrade to 6.01 helps you, too!