by Andrew Johnstone
I Spent a little while yesterday investigating why memcached causes problems with multigets returning results and ended up comparing debian vs ubuntu. I tested from virtual machines and physical hardware of relative specs. I have also tried a combination of libmemcached libraries and tuned the tcp stack, however here are some of the results from the tests.
A larger number of keys than 200 causes memcached to fail to return valid responses, additionally the key length greatly varies the number items you can pull back within a single multiget on Ubuntu, Debian implementation of memcached is able to handle all requests regardless of key length size.
Tuning sysctl does not make much difference in the results.
A quick test.
<?php
$m = new Memcached();
$m->addServer('localhost', 11211);
function _log($msg) {
print date('Y-m-d H:i:s').": {$msg}\n";
}
_log("Generating");
$data3 = array();
for ($i=0; $i< =200000; $i++) {
$data3[md5(microtime(true) . rand(0,10100000))]=111111111111;
}
_log("Generated");
$m->setMulti($data3);
_log("Result Code from setMulti: ".$m->getResultCode());
$i=10000;
do {
_log("Getting Slice smaller key ({$i})");
$result = $m->getMulti(array_slice(array_keys($data3),0,$i), $cas);
_log("Result cnt: ". count($result) . " Cas cnt:" . count($cas) . " Result code:" . $m->getResultCode());
$i-=500;
if ($i< =500) {
$i+=500;
$i-=100;
}
if ($i<=200) {
$i+=100;
$i-=10;
}
sleep(3);
} while($i>0);
andrew@andrew-desktop:~/Downloads$ php test.php 2011-06-07 13:19:26: Generating 2011-06-07 13:19:37: Generated 2011-06-07 13:19:47: Result Code from setMulti: 0 2011-06-07 13:19:47: Getting Slice smaller key (10000) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:19:47: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:19:50: Getting Slice smaller key (9500) 2011-06-07 13:19:50: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:19:53: Getting Slice smaller key (9000) 2011-06-07 13:19:53: Result cnt: 60 Cas cnt:60 Result code:0 2011-06-07 13:19:56: Getting Slice smaller key (8500) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:19:56: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:19:59: Getting Slice smaller key (8000) 2011-06-07 13:19:59: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:20:02: Getting Slice smaller key (7500) 2011-06-07 13:20:02: Result cnt: 50 Cas cnt:50 Result code:0 2011-06-07 13:20:05: Getting Slice smaller key (7000) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:20:05: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:20:08: Getting Slice smaller key (6500) 2011-06-07 13:20:09: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:20:12: Getting Slice smaller key (6000) 2011-06-07 13:20:12: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:20:15: Getting Slice smaller key (5500) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:20:15: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:20:18: Getting Slice smaller key (5000) 2011-06-07 13:20:18: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:20:21: Getting Slice smaller key (4500) 2011-06-07 13:20:21: Result cnt: 30 Cas cnt:30 Result code:0 2011-06-07 13:20:24: Getting Slice smaller key (4000) 2011-06-07 13:20:24: Result cnt: 26 Cas cnt:26 Result code:0 2011-06-07 13:20:27: Getting Slice smaller key (3500) 2011-06-07 13:20:27: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:20:30: Getting Slice smaller key (3000) 2011-06-07 13:20:30: Result cnt: 20 Cas cnt:20 Result code:0 2011-06-07 13:20:33: Getting Slice smaller key (2500) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:20:33: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:20:36: Getting Slice smaller key (2000) 2011-06-07 13:20:36: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:20:39: Getting Slice smaller key (1500) 2011-06-07 13:20:39: Result cnt: 10 Cas cnt:10 Result code:0 2011-06-07 13:20:42: Getting Slice smaller key (1000) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:20:43: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:20:46: Getting Slice smaller key (900) 2011-06-07 13:20:46: Result cnt: 155 Cas cnt:155 Result code:0 2011-06-07 13:20:49: Getting Slice smaller key (800) 2011-06-07 13:20:49: Result cnt: 55 Cas cnt:55 Result code:0 2011-06-07 13:20:52: Getting Slice smaller key (700) 2011-06-07 13:20:52: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:20:55: Getting Slice smaller key (600) 2011-06-07 13:20:55: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:20:58: Getting Slice smaller key (500) 2011-06-07 13:20:58: Result cnt: 0 Cas cnt:0 Result code:19 2011-06-07 13:21:01: Getting Slice smaller key (400) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:21:01: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:21:04: Getting Slice smaller key (300) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:21:04: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:21:07: Getting Slice smaller key (290) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:21:07: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:21:10: Getting Slice smaller key (280) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:21:10: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:21:13: Getting Slice smaller key (270) 2011-06-07 13:21:14: Result cnt: 270 Cas cnt:270 Result code:0 2011-06-07 13:21:17: Getting Slice smaller key (260) WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv() 2011-06-07 13:21:17: Result cnt: 0 Cas cnt:0 Result code:0 2011-06-07 13:21:20: Getting Slice smaller key (250) 2011-06-07 13:21:20: Result cnt: 250 Cas cnt:250 Result code:0 2011-06-07 13:21:23: Getting Slice smaller key (240) 2011-06-07 13:21:23: Result cnt: 240 Cas cnt:240 Result code:0 2011-06-07 13:21:26: Getting Slice smaller key (230) 2011-06-07 13:21:26: Result cnt: 230 Cas cnt:230 Result code:0 2011-06-07 13:21:29: Getting Slice smaller key (220) 2011-06-07 13:21:29: Result cnt: 220 Cas cnt:220 Result code:0 2011-06-07 13:21:32: Getting Slice smaller key (210) 2011-06-07 13:21:32: Result cnt: 210 Cas cnt:210 Result code:0 2011-06-07 13:21:35: Getting Slice smaller key (200) 2011-06-07 13:21:35: Result cnt: 200 Cas cnt:200 Result code:0 2011-06-07 13:21:38: Getting Slice smaller key (190) 2011-06-07 13:21:38: Result cnt: 190 Cas cnt:190 Result code:0 2011-06-07 13:21:41: Getting Slice smaller key (180) 2011-06-07 13:21:41: Result cnt: 180 Cas cnt:180 Result code:0 2011-06-07 13:21:44: Getting Slice smaller key (170) 2011-06-07 13:21:44: Result cnt: 170 Cas cnt:170 Result code:0 2011-06-07 13:21:47: Getting Slice smaller key (160) 2011-06-07 13:21:48: Result cnt: 160 Cas cnt:160 Result code:0 2011-06-07 13:21:51: Getting Slice smaller key (150) 2011-06-07 13:21:51: Result cnt: 150 Cas cnt:150 Result code:0 2011-06-07 13:21:54: Getting Slice smaller key (140) 2011-06-07 13:21:54: Result cnt: 140 Cas cnt:140 Result code:0 2011-06-07 13:21:57: Getting Slice smaller key (130) 2011-06-07 13:21:57: Result cnt: 130 Cas cnt:130 Result code:0 2011-06-07 13:22:00: Getting Slice smaller key (120) 2011-06-07 13:22:00: Result cnt: 120 Cas cnt:120 Result code:0 2011-06-07 13:22:03: Getting Slice smaller key (110) 2011-06-07 13:22:03: Result cnt: 110 Cas cnt:110 Result code:0 2011-06-07 13:22:06: Getting Slice smaller key (100) 2011-06-07 13:22:06: Result cnt: 100 Cas cnt:100 Result code:0 2011-06-07 13:22:09: Getting Slice smaller key (90) 2011-06-07 13:22:09: Result cnt: 90 Cas cnt:90 Result code:0 2011-06-07 13:22:12: Getting Slice smaller key (80) 2011-06-07 13:22:12: Result cnt: 80 Cas cnt:80 Result code:0 2011-06-07 13:22:15: Getting Slice smaller key (70) 2011-06-07 13:22:16: Result cnt: 70 Cas cnt:70 Result code:0 2011-06-07 13:22:19: Getting Slice smaller key (60) 2011-06-07 13:22:19: Result cnt: 60 Cas cnt:60 Result code:0 2011-06-07 13:22:22: Getting Slice smaller key (50) 2011-06-07 13:22:22: Result cnt: 50 Cas cnt:50 Result code:0 2011-06-07 13:22:25: Getting Slice smaller key (40) 2011-06-07 13:22:25: Result cnt: 40 Cas cnt:40 Result code:0 2011-06-07 13:22:28: Getting Slice smaller key (30) 2011-06-07 13:22:28: Result cnt: 30 Cas cnt:30 Result code:0 2011-06-07 13:22:31: Getting Slice smaller key (20) 2011-06-07 13:22:31: Result cnt: 20 Cas cnt:20 Result code:0 2011-06-07 13:22:34: Getting Slice smaller key (10) 2011-06-07 13:22:34: Result cnt: 10 Cas cnt:10 Result code:0
andrew@bb1:~$ php test.php 2011-06-07 13:20:38: Generating 2011-06-07 13:20:41: Generated 2011-06-07 13:20:52: Result Code from setMulti: 0 2011-06-07 13:20:52: Getting Slice smaller key (10000) 2011-06-07 13:20:53: Result cnt: 10000 Cas cnt:10000 Result code:0 2011-06-07 13:20:56: Getting Slice smaller key (9500) 2011-06-07 13:20:56: Result cnt: 9500 Cas cnt:9500 Result code:0 2011-06-07 13:20:59: Getting Slice smaller key (9000) 2011-06-07 13:20:59: Result cnt: 9000 Cas cnt:9000 Result code:0 2011-06-07 13:21:02: Getting Slice smaller key (8500) 2011-06-07 13:21:02: Result cnt: 8500 Cas cnt:8500 Result code:0 2011-06-07 13:21:05: Getting Slice smaller key (8000) 2011-06-07 13:21:05: Result cnt: 8000 Cas cnt:8000 Result code:0 2011-06-07 13:21:08: Getting Slice smaller key (7500) 2011-06-07 13:21:09: Result cnt: 7500 Cas cnt:7500 Result code:0 2011-06-07 13:21:12: Getting Slice smaller key (7000) 2011-06-07 13:21:12: Result cnt: 7000 Cas cnt:7000 Result code:0 2011-06-07 13:21:15: Getting Slice smaller key (6500) 2011-06-07 13:21:15: Result cnt: 6500 Cas cnt:6500 Result code:0 2011-06-07 13:21:18: Getting Slice smaller key (6000) 2011-06-07 13:21:18: Result cnt: 6000 Cas cnt:6000 Result code:0 2011-06-07 13:21:21: Getting Slice smaller key (5500) 2011-06-07 13:21:22: Result cnt: 5500 Cas cnt:5500 Result code:0 2011-06-07 13:21:25: Getting Slice smaller key (5000) 2011-06-07 13:21:25: Result cnt: 5000 Cas cnt:5000 Result code:0 2011-06-07 13:21:28: Getting Slice smaller key (4500) 2011-06-07 13:21:28: Result cnt: 4500 Cas cnt:4500 Result code:0 2011-06-07 13:21:31: Getting Slice smaller key (4000) 2011-06-07 13:21:31: Result cnt: 4000 Cas cnt:4000 Result code:0 2011-06-07 13:21:34: Getting Slice smaller key (3500) 2011-06-07 13:21:34: Result cnt: 3500 Cas cnt:3500 Result code:0 2011-06-07 13:21:37: Getting Slice smaller key (3000) 2011-06-07 13:21:38: Result cnt: 3000 Cas cnt:3000 Result code:0 2011-06-07 13:21:41: Getting Slice smaller key (2500) 2011-06-07 13:21:41: Result cnt: 2500 Cas cnt:2500 Result code:0 2011-06-07 13:21:44: Getting Slice smaller key (2000) 2011-06-07 13:21:44: Result cnt: 2000 Cas cnt:2000 Result code:0 2011-06-07 13:21:47: Getting Slice smaller key (1500) 2011-06-07 13:21:47: Result cnt: 1500 Cas cnt:1500 Result code:0 2011-06-07 13:21:50: Getting Slice smaller key (1000) 2011-06-07 13:21:50: Result cnt: 1000 Cas cnt:1000 Result code:0 2011-06-07 13:21:53: Getting Slice smaller key (900) 2011-06-07 13:21:54: Result cnt: 900 Cas cnt:900 Result code:0 2011-06-07 13:21:57: Getting Slice smaller key (800) 2011-06-07 13:21:57: Result cnt: 800 Cas cnt:800 Result code:0 2011-06-07 13:22:00: Getting Slice smaller key (700) 2011-06-07 13:22:00: Result cnt: 700 Cas cnt:700 Result code:0 2011-06-07 13:22:03: Getting Slice smaller key (600) 2011-06-07 13:22:03: Result cnt: 600 Cas cnt:600 Result code:0 2011-06-07 13:22:06: Getting Slice smaller key (500) 2011-06-07 13:22:06: Result cnt: 500 Cas cnt:500 Result code:0 2011-06-07 13:22:09: Getting Slice smaller key (400) 2011-06-07 13:22:10: Result cnt: 400 Cas cnt:400 Result code:0 2011-06-07 13:22:13: Getting Slice smaller key (300) 2011-06-07 13:22:13: Result cnt: 300 Cas cnt:300 Result code:0 2011-06-07 13:22:16: Getting Slice smaller key (290) 2011-06-07 13:22:16: Result cnt: 290 Cas cnt:290 Result code:0 2011-06-07 13:22:19: Getting Slice smaller key (280) 2011-06-07 13:22:19: Result cnt: 280 Cas cnt:280 Result code:0 2011-06-07 13:22:22: Getting Slice smaller key (270) 2011-06-07 13:22:22: Result cnt: 270 Cas cnt:270 Result code:0 2011-06-07 13:22:25: Getting Slice smaller key (260) 2011-06-07 13:22:26: Result cnt: 260 Cas cnt:260 Result code:0 2011-06-07 13:22:29: Getting Slice smaller key (250) 2011-06-07 13:22:29: Result cnt: 250 Cas cnt:250 Result code:0 2011-06-07 13:22:32: Getting Slice smaller key (240) 2011-06-07 13:22:32: Result cnt: 240 Cas cnt:240 Result code:0 2011-06-07 13:22:35: Getting Slice smaller key (230) 2011-06-07 13:22:35: Result cnt: 230 Cas cnt:230 Result code:0 2011-06-07 13:22:38: Getting Slice smaller key (220) 2011-06-07 13:22:38: Result cnt: 220 Cas cnt:220 Result code:0 2011-06-07 13:22:41: Getting Slice smaller key (210) 2011-06-07 13:22:41: Result cnt: 210 Cas cnt:210 Result code:0 2011-06-07 13:22:44: Getting Slice smaller key (200) 2011-06-07 13:22:44: Result cnt: 200 Cas cnt:200 Result code:0 2011-06-07 13:22:47: Getting Slice smaller key (190) 2011-06-07 13:22:48: Result cnt: 190 Cas cnt:190 Result code:0 2011-06-07 13:22:51: Getting Slice smaller key (180) 2011-06-07 13:22:51: Result cnt: 180 Cas cnt:180 Result code:0 2011-06-07 13:22:54: Getting Slice smaller key (170) 2011-06-07 13:22:54: Result cnt: 170 Cas cnt:170 Result code:0 2011-06-07 13:22:57: Getting Slice smaller key (160) 2011-06-07 13:22:57: Result cnt: 160 Cas cnt:160 Result code:0 2011-06-07 13:23:00: Getting Slice smaller key (150) 2011-06-07 13:23:00: Result cnt: 150 Cas cnt:150 Result code:0 2011-06-07 13:23:03: Getting Slice smaller key (140) 2011-06-07 13:23:03: Result cnt: 140 Cas cnt:140 Result code:0 2011-06-07 13:23:06: Getting Slice smaller key (130) 2011-06-07 13:23:07: Result cnt: 130 Cas cnt:130 Result code:0 2011-06-07 13:23:10: Getting Slice smaller key (120) 2011-06-07 13:23:10: Result cnt: 120 Cas cnt:120 Result code:0 2011-06-07 13:23:13: Getting Slice smaller key (110) 2011-06-07 13:23:13: Result cnt: 110 Cas cnt:110 Result code:0 2011-06-07 13:23:16: Getting Slice smaller key (100) 2011-06-07 13:23:16: Result cnt: 100 Cas cnt:100 Result code:0 2011-06-07 13:23:19: Getting Slice smaller key (90) 2011-06-07 13:23:19: Result cnt: 90 Cas cnt:90 Result code:0 2011-06-07 13:23:22: Getting Slice smaller key (80) 2011-06-07 13:23:22: Result cnt: 80 Cas cnt:80 Result code:0 2011-06-07 13:23:25: Getting Slice smaller key (70) 2011-06-07 13:23:25: Result cnt: 70 Cas cnt:70 Result code:0 2011-06-07 13:23:28: Getting Slice smaller key (60) 2011-06-07 13:23:29: Result cnt: 60 Cas cnt:60 Result code:0 2011-06-07 13:23:32: Getting Slice smaller key (50) 2011-06-07 13:23:32: Result cnt: 50 Cas cnt:50 Result code:0 2011-06-07 13:23:35: Getting Slice smaller key (40) 2011-06-07 13:23:35: Result cnt: 40 Cas cnt:40 Result code:0 2011-06-07 13:23:38: Getting Slice smaller key (30) 2011-06-07 13:23:38: Result cnt: 30 Cas cnt:30 Result code:0 2011-06-07 13:23:41: Getting Slice smaller key (20) 2011-06-07 13:23:41: Result cnt: 20 Cas cnt:20 Result code:0 2011-06-07 13:23:44: Getting Slice smaller key (10) 2011-06-07 13:23:44: Result cnt: 10 Cas cnt:10 Result code:0
I have been a developer for roughly 10 years and have worked with an extensive range of technologies. Whilst working for relatively small companies, I have worked with all aspects of the development life cycle, which has given me a broad and in-depth experience.
2 Responses to Memcached multigets – ubuntu vs debian
Tomasz Sh4dow Budzyński
June 15th, 2011 at 7:35 am
I think You should check your Ubuntu. On my Kubuntu i’m not getting any errors and all work fine.
Kubuntu 11.04
PHP 5.3.5-1ubuntu7.2 with Suhosin-Patch (cli) (built: May 2 2011 23:00:17)
Andrew Johnstone’s Blog: Memcached multigets – ubuntu vs debian | PHP Boutique
June 15th, 2011 at 7:39 am
[…] er memcached multigets mit einer großen Anzahl von Schlüsseln verwenden wollte – es war werfen Ausfälle , aber nur wenn er es versucht auf Ubuntu. Ich verbrachte eine Weile gestern erforschen, warum […]