Thursday, August 04, 2011

ORA 600 [kglLockOwnersListAppend-ovf]

On 11gR2 RAC, I found error:
ORA-00600: internal error code, arguments: [kglLockOwnersListAppend-ovf]
That's so bad, that made CPU over load!!! In alert log file.
Errors in file /u01/app/oracle/diag/rdbms/exadb/exadb3/trace/exadb3_ora_2731.trc (incident=105200):
ORA-00600: internal error code, arguments: [kglLockOwnersListAppend-ovf], [0xF37199BF8], [0xDA8EE6DD0], [], [], [], [], [], [], [], [], []
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Checked in exadb3_ora_2731.trc file:
DDE: Problem Key 'ORA 600 [kglLockOwnersListAppend-ovf]' was flood controlled (0x2) (incident: 105200)
ORA-00600: internal error code, arguments: [kglLockOwnersListAppend-ovf], [0xF37199BF8], [0xDA8EE6DD0], [], [], [], [], [], [], [], [], []

LibraryHandle: Address=0xf37199bf8 Hash=e3879740 LockMode=N PinMode=0 LoadLockMode=0 Status=VALD
ObjectName: Name=SELECT ...

FullHashValue=c6908e795c6d3f7597bfeb43e3879740 Namespace=SQL AREA(00) Type=CURSOR(00) Identifier=3817314112 OwnerIdn=99
Statistics: InvalidationCount=0 ExecutionCount=2097399 LoadCount=6 ActiveLocks=65535 TotalLockCount=2700711 TotalPinCount=2
Counters: BrokenCount=1 RevocablePointer=1 KeepDependency=2 BucketInUse=5923 HandleInUse=5923 HandleReferenceCount=0
Concurrency: DependencyMutex=0xf37199ca8(0, 1164076, 10199, 0) Mutex=0xf37199d28(949, 10513971, 75334639, 6)
Flags=RON/PIN/TIM/PN0/DBN/[10012841]
Then checked on My Oracle Support. It's bug - -" and checked NOTE: Frequently Getting ORA-600 [kglLockOwnersListAppend-ovf] Or ORA-600 [kglLockOwnersListDelete] [ID 1318902.1]

This case causes from rowcache cursor and session_cached_cursors, that is set to an extremely high value. I set session_cached_cursors (high value), then each session would end up opening a lot of duplicate cursors. That caused one library cache lock to be held, it is very easy to exceed the limit of 64K locks (65535).

After I found out more. I can use Patch 10086843, but I set SESSION_CACHED_CURSORS=0 for workaround before.
Some Idea, .... session_cached_cursors * MaxCurrentSessions < 65535

Reference:
- SESSION_CACHED_CURSORS It causes the session to keep a cache of recently closed cursors.
- My Oracle Support

No comments: