Performance Testing with Agile Data

Performance testing requires full, fresh data

Many organizations don’t even attempt to test performance until very late in their development cycle because it is only in the UAT (or equivalent) environment that they have a full copy of their production data set.  Errors and inefficiencies found at this stage are expensive to fix and are often promoted to production due to pressures from the business to meet release schedules.

Delphix customers give each developer, or team of developers, a full, fresh copy of the database where they can validate the performance of their code in the UNIT TESTING phase of their projects.  Poorly written queries are identified by the developer and can be fixed immediately, before their code is submitted for integration with the rest of the application.   The test/fix iteration is much tighter and results in higher quality, better performing application releases.

How does Delphix enable this?

VDBs created by Delphix have many advantages over a physical database, and therefore can be used in unique ways.  Consider the following:

• Self service.  Delphix automates all the complexity required to make changes to a database, allowing developers and testers to get what they need without waiting on associated support organizations.
• Fast provisioning.  VDBs require no data movement at the time they are created, so even large databases can be created in a few minutes.
• Easy data refresh.  Refreshing a VDB with the latest data from production can be done with 3 mouse clicks.  Never test against synthetic data.
• Data rewind/reset.  Delphix tracks all changes made to a VDB and can rewind the state of the database to any point in time at the request of the user.  Run a test, rewind, change parameters, run the test again.
• Efficient use of infrastructure.  VDBs run in a tiny storage footprint, allowing teams to run many more database environments in parallel.
• Efficient use of licenses.  Turning VDBs on and off is trivial.  Test environments can be spun up as needed and suspended when testing is finished.  Suspended VDBs use no resources of the DBMS.
• Database relocation.  VDBs are easily moved between database hosts, even across datacenters.

Following are examples of performance changes that can easily be tested in VDBs:

Database Configuration

• changes to initialization parameters (eg. optimizer_index_cost_adj, optimizer_index_caching, etc)
• changes to redo size, parameters
• DBMS version and patch set
• SGA size
• CPU type, speed (move VDB between database hosts)
• Different DB statistics, statistics gathering methods

Data Modeling

• Index changes
• SQL Profiles – Like the old stored outlines, you can set up a complex system of profiles on a VDB and test different explain plans
• Run complex and potentially debilitating queries on a VDB to minimize impact, use TKPROF and heavy tracing you can’t do elsewhere

Application  Configuration

• Testing application server connection pool sizes/limits
• network bandwidth testing for multi-hop/firewall configuration
• theoretical maximums for concurrent batch jobs (not just at the DB, but the app tier as well)
• testing database monitoring solutions/thresholds/configuration impact
• Oracle trace event impact when turned on (deviation from a baseline)

Enabling a physical UAT environment with Delphix

As mentioned above, many Delphix customers will still maintain a final testing environment that matches the production setup exactly.  They will have fibre channel (or equivalent) connections to the SAN directly from their DBMS host.  Even in this environment, which bypasses the Delphix Engine, our software can provide great benefit to the testing process.

The V2P feature can be used to migrate any data set contained within Delphix to a physical storage environment.  That means any data set captured from production, or any data set modified by a VDB can be pushed to UAT in an automated fashion by Delphix.  Running a V2P operation is not as fast as creating/refreshing a VDB because it requires data movement, but it is faster than restoring a traditional database backup and automates all the instance creation and configuration tasks.

Bringing it all together

The high level life cycle of performance testing on the Delphix Agile Data Platform looks something like the following:

1. Create and/or refresh development environments with the latest full data set from production.
2. Use VDBs to iterate quickly on unit tests of new code, data modeling changes, DBMS configuration changes.
3. Integrate and test all changes in a highly parallelized QA environment, using VDBs to minimize the setup time between test runs.
4. Run V2P to migrate release candidates to UAT for final performance verification.
5. Promote changes to production.

 

A