Error: ORA-16810: multiple errors or warnings detected

Today after restarting primary and standby database servers I faced with ORA-16810 error

DGMGRL> show configuration;

Configuration - DGMANAGER

  Protection Mode: MaxAvailability
  Databases:
    XXXPROD - Primary database
      Error: ORA-16810: multiple errors or warnings detected for the database

    XXXSTND - Physical standby database
      Error: ORA-01017: invalid username/password; logon denied

Fast-Start Failover: DISABLED

Configuration Status:
ERROR

I did not understand what and why it happened...
After research I found that it was related password file of standby side.


DGMGRL> show database 'XXXPROD'  

Database - XXXPROD

  Role:            PRIMARY
  Intended State:  TRANSPORT-ON
  Instance(s):
    FCDBPROD
      Error: ORA-16737: the redo transport service for standby database "XXXSTND" has an error

  Database Warning(s):
    ORA-16629: database reports a different protection level from the protection mode

Database Status:
ERROR

--On standby side
DGMGRL> show configuration

Configuration - DGMANAGER

  Protection Mode: MaxAvailability
  Databases:
    XXXPROD - Primary database
    XXXSTND - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
ORA-01017: invalid username/password; logon denied
ORA-16625: cannot reach database "XXXPROD"
DGM-17017: unable to determine configuration status


But connection between servers were clean. Listeners were up and tnsping success.

Hm, interesting what will provide us v$archive_dest:

SQL> select error from v$archive_dest;

ERROR
-----------------------------------------------------------------

ORA-16191: Primary log shipping client not logged on standby

It looks like could not connect to my standby db. Both instance are up and tnsping were OK, Also using
sqlplus user/pass@'connection_stringPRIMARY'
sqlplus user/pass@'connection_stringSTANDBY'

on both side between were success.


On the forum I found that something related passwordfile.
But I did not change any thing...

On both side remote_login_passwordfile`s value was EXCLUSIVE

SQL> show parameter remote_login_passwordfile

NAME     TYPE VALUE
------------------------------------ ----------- ------------------------------
remote_login_passwordfile     string EXCLUSIVE
SQL>

And

SQL> select * from v$pwfile_users;

USERNAME       SYSDB SYSOP SYSAS
------------------------------ ----- ----- -----
SYS       TRUE  TRUE  FALSE

SQL>

I just copied Primary`s password file to standby side and then checked.
before it renamed old password file: mv orapwXXX.old

scp orapwXXX oracle@XX.XX.XX.X:/u01/app/oracle/product/11.2.0.3/db_1/dbs
--renamed it also for stanby



now everything is OK

[oracle@fcdbdb dbs]$ dgmgrl /
DGMGRL for Linux: Version 11.2.0.3.0 - 64bit Production

Copyright (c) 2000, 2009, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.
Connected.
DGMGRL> show configuration

Configuration - DGMANAGER

  Protection Mode: MaxAvailability
  Databases:
    XXXPROD - Primary database
    XXXSTND - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS

--primary
SQL>  select max(sequence#) from v$archived_log;

MAX(SEQUENCE#)
--------------
 7819

--standby
SQL>  select max(sequence#) from v$archived_log;

MAX(SEQUENCE#)
--------------
 7819