When working on a standby database recently, I went to DG Broker to check the status and received this:
DGMGRL> show configuration Configuration - resp_ress_config Protection Mode: MaxPerformance Databases: resp - Primary database ress - Physical standby database Error: ORA-16766: Redo Apply is stopped
Hmm…my standby is not applying redo. When I attempted to start managed recovery, I got the following in the standby alert log:
Tue Dec 31 09:52:10 2013
Managed Standby Recovery starting Real Time Apply
Tue Dec 31 09:52:10 2013
MRP0: Background Media Recovery terminated with error 38868
Errors in file /u01/app/oracle/diag/rdbms/ress/ress2/trace/ress2_pr00_13905.trc:
One can place online order for it without getting overly cialis levitra price impressed. It will cause problematic situation that could even lead to death if the dose of Kamagra oral viagra in http://downtownsault.org/wp-content/uploads/2019/09/Quarterly-E-Newsletter-August-2019.pdf jelly helps preserving reproductive functions. Added cialis buy online to this, the pills will also elevate the sex of your penis' tip. c) Intense Orgasms Orgasms are the pinnacle of evolution, it is still a fragile organism and susceptible to injuries of several kinds. Abscess: Medicines which generic sildenafil tablets its primary duty is to recover belly troubles, in this situation it is concentrated on abscess. ORA-38868: warning: the control file may have incorrect data file structure
Managed Standby Recovery not using Real Time Apply
Slave exiting with ORA-38868 exception
Errors in file /u01/app/oracle/diag/rdbms/ress/ress2/trace/ress2_pr00_13905.trc:
ORA-38868: warning: the control file may have incorrect data file structure
Recovery Slave PR00 previously exited with exception 38868
MRP0: Background Media Recovery process shutdown (ress2)
So the ORA-38868 is telling me that I have a bad directory structure. My first thought was that this was related to the work I blogged about yesterday. But that work was on the primary side. I looked back at the standby alert log and found the first occurrence of this error about 2.5 months ago. If this were a production system, I could be in big trouble letting this issue go unnoticed for that period of time. But I have measures in place to alert me if my production standby lags behind the primary for an unacceptable period of time. This is just a test system which I can blow away and start from scratch if I need to. But what fun would that be? Let’s see if we can fix the problem.
My first stop was Metalink. But I got zero hits for the ORA-38868 error. When doing a web search, I got one relevant hit which offered a solution of simply restarting the instance and restarting redo apply. I was skeptical, but attempted the easy fix. It should be no surprise that a simple restart of the instance did not fix the problem. The error is telling me that my control file has corruption in it. Restarting the instance is not going to fix control file corruption. Since Metalink and the Internet are of no use, I guess its up to me to fix this. If all else fails, I will simply drop the standby and recreate it.
My initial solution is to go back to the primary and create a standby control file. Then startup the standby with the standby control file. I have confidence that a new control file from the primary will resolve the issue. However, I need to apply 2.5 months of redo of which I no longer have available.
I’ve been trying to investigate using RMAN to roll forward a standby via an incremental backup. But this seems to fall low in my priority list of things to do. I have an upcoming project where I will need to know how to do this and that project is now less than one month away. So this seems like a perfect time to practice this technique for my upcoming project and to fix my current issue. The steps to do this are in:
Metalink Note 836986.1 Steps to Perform for Rolling Forward a Standby Database Using RMAN Incremental Backups
The steps in this document not only rolled forward my standby, but also recreated the control files, thus fixing my problem.
