I'm having a problem with apache (lamp). Once in a while i have a process that uses up one of my cpu's for like 5-20 secs.
Normally it would be php's fault, but after testing i could see that the php script finished. (at the final echo command) but just apache was waiting for something. I have a blazing fast server with memory to spare, harddrives (300+Mb/s) almost doing nothing and less then 10% cpu utilization under high traffic.
below i have a dump from dstat, as you can see i have ~12 cpu usage for about 20 secs on the sys column, this happens when apache takes up one cpu.
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
7 12 81 0 0 0|8192B 0 | 16k 96k| 0 0 |1302 3030
8 12 80 0 0 0| 16k 0 | 18k 99k| 0 0 |1297 2715
10 12 77 0 0 0| 248k 1384k| 24k 206k| 0 0 |1501 3857
15 12 73 0 0 0| 40k 0 | 36k 459k| 0 0 |1574 3643
9 12 79 1 0 0| 208k 4024k| 42k 427k| 0 0 |1865 3476
5 12 83 0 0 0| 56k 920k| 40k 574k| 0 0 |1648 2137
5 12 82 1 0 0| 160k 0 | 30k 275k| 0 0 |1556 2776
6 12 82 0 0 0| 96k 1776k| 23k 198k| 0 0 |1354 2811
3 12 85 0 0 0| 24k 0 | 18k 132k| 0 0 |1418 2200
5 12 83 0 0 0| 24k 3288k| 28k 263k| 0 0 |1457 2422
7 12 80 1 0 0| 176k 1080k| 29k 310k| 0 0 |1616 4206
5 12 83 1 0 0| 312k 0 | 63k 639k| 0 0 |1944 2776
4 12 73 10 0 0| 96k 8912k| 33k 366k| 0 0 |2652 2628
8 12 80 0 0 0| 56k 0 | 26k 232k| 0 0 |1401 2909
4 12 71 14 0 0| 88k 0 | 21k 178k| 0 0 |1492 3041
8 12 78 2 0 0| 144k 3984k| 61k 677k| 0 0 |1971 3893
5 12 83 0 0 0| 104k 0 | 35k 439k| 0 0 |1650 2878
5 12 81 1 0 0| 112k 1352k| 31k 210k| 0 0 |1476 3052
5 12 84 0 0 0|8192B 1792k| 15k 102k| 0 0 |1387 2663
4 13 81 2 0 0| 936k 4456k| 26k 162k| 0 0 |1491 2644
6 12 81 1 0 0| 200k 1336k| 33k 315k| 0 0 |1676 2829
7 15 77 1 0 0| 96k 672k| 37k 337k| 0 0 |1549 2758
9 13 77 1 0 0| 104k 0 | 51k 399k| 0 0 |1871 3614
5 12 82 0 0 0| 72k 2392k| 46k 427k| 0 0 |1685 2803
7 11 82 0 0 0| 64k 0 | 21k 214k| 0 0 |1487 2762
7 0 93 0 0 0| 32k 0 | 21k 155k| 0 0 |1310 2739
5 0 95 0 0 0| 32k 0 | 40k 333k| 0 0 |1675 2829
so how can i debug this? or what could it be?