A Java service's memory grows until the container is OOM-killed every two days. How do you diagnose it?
Short answer
Determine whether heap or native memory grows using GC logs, metrics and Native Memory Tracking. For heap growth, take heap dumps at intervals and analyse retained objects (dominator tree) to find the leak; for native growth, check direct buffers, thread counts, metaspace and native libraries; then fix and set safe container sizing.
Detailed answer
- Heap vs non-heap: compare heap used after GC (from GC logs/JMX) with container RSS.
- Heap leak: old generation after full GCs keeps rising →
jcmd <pid> GC.heap_dumptwice, compare in Eclipse MAT; common culprits: unbounded caches/maps, listeners not removed, ThreadLocals, session data. - Native growth: enable
-XX:NativeMemoryTracking=summary; check thread count (stack per thread), direct ByteBuffers (Netty), metaspace (class loading leaks from repeated deployments/dynamic proxies), JNI libraries; glibc malloc fragmentation (tryMALLOC_ARENA_MAX). - Fix and safeguards: bounded caches (Caffeine with max size),
MaxRAMPercentageleaving headroom,-XX:+ExitOnOutOfMemoryErrorand heap dump on OOM, memory alerts.
What the interviewer expects
A methodical separation of heap and native memory and concrete tools.
Common mistakes to avoid
Just increasing the memory limit.