100% CPU usage

You are here

100% CPU usage

5 posts / 0 new
Last post
uniks
uniks's picture
100% CPU usage

Hello,

From time to time our as2 starts eating 100% cpu. This has happened in b25 and the issue is still in b27. I can't see any relation between each occurrence. Sometimes this happens during the night when there are no connections. It can happen after one day or after month from the AS2 start.

Linux, RH 4.4 ES
mec_AS b27

Does anyone observe something similar ?

heller
heller's picture

uniks,

There are some third party products included in the AS2 server that may start threads, perhaps this is a database reorganization or something like this.

Regards
Heller

uniks
uniks's picture

Hi again,

I'm still fighting with this issue,
currently I'm able to reproduce this by executing:

for i in `seq 1 1000`; do `wget http://sv-as2:8888/as2/ > /dev/null 2>&1`; done &
for i in `seq 1 1000`; do `wget http://sv-as2:8888/as2/ > /dev/null 2>&1`; done &

so, 2k request for index.html, this caused that AS2 starts using 100% cpu, even if all request finished.
I tried switch AS2 from sun java to jrockit, but the problem is also there. Using jrmc I trace two threads consuming this cpu:

Stack Trace
btpool0-7 - Acceptor1 SelectChannelConnector @ 0.0.0.0:8080 [36] (RUNNABLE)
java.lang.System.currentTimeMillis line: not available [native method]
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect line: 312
org.mortbay.io.nio.SelectorManager.doSelect line: 73
org.mortbay.jetty.nio.SelectChannelConnector.accept line: 120
org.mortbay.jetty.AbstractConnector$Acceptor.run line: 498
org.mortbay.thread.BoundedThreadPool$PoolThread.run line: 475

Stack Trace
btpool0-7 - Acceptor1 SelectChannelConnector @ 0.0.0.0:8080 [36] (RUNNABLE)
sun.nio.ch.EPollArrayWrapper.epollWait line: not available [native method]
sun.nio.ch.EPollArrayWrapper.poll line: 215
sun.nio.ch.EPollSelectorImpl.doSelect line: 65
sun.nio.ch.SelectorImpl.lockAndDoSelect line: 69
sun.nio.ch.SelectorImpl.select line: 80
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect line: 299
org.mortbay.io.nio.SelectorManager.doSelect line: 73
org.mortbay.jetty.nio.SelectChannelConnector.accept line: 120
org.mortbay.jetty.AbstractConnector$Acceptor.run line: 498
org.mortbay.thread.BoundedThreadPool$PoolThread.run line: 475

PS.
I tested this in b25, b27, b29
OS: Linux RH 4.4
tested with sun java jre1.6.0_11 and jrockit jrrt-4.0.0-1.6.0

uniks
uniks's picture

miss click, the second thread was:

Stack Trace
btpool0-8 - Acceptor0 SelectChannelConnector @ 0.0.0.0:8080 [37] (RUNNABLE)
java.lang.System$2.blockedOn line: 1143
java.nio.channels.spi.AbstractInterruptibleChannel.blockedOn line: 191
java.nio.channels.spi.AbstractSelector.end line: 210
sun.nio.ch.EPollSelectorImpl.doSelect line: 67
sun.nio.ch.SelectorImpl.lockAndDoSelect line: 69
sun.nio.ch.SelectorImpl.select line: 80
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect line: 299
org.mortbay.io.nio.SelectorManager.doSelect line: 73
org.mortbay.jetty.nio.SelectChannelConnector.accept line: 120
org.mortbay.jetty.AbstractConnector$Acceptor.run line: 498
org.mortbay.thread.BoundedThreadPool$PoolThread.run line: 475

heller
heller's picture

uniks,

please update to the newest version of the mendelson opensource AS2 server.

Regards
Heller