Memcached multigets – ubuntu vs debian

In: General|Linux|PHP

14 Jun 2011

I Spent a little while yesterday investigating why memcached causes problems with multigets returning results and ended up comparing debian vs ubuntu. I tested from virtual machines and physical hardware of relative specs. I have also tried a combination of libmemcached libraries and tuned the tcp stack, however here are some of the results from the tests.

A larger number of keys than 200 causes memcached to fail to return valid responses, additionally the key length greatly varies the number items you can pull back within a single multiget on Ubuntu, Debian implementation of memcached is able to handle all requests regardless of key length size.

Tuning sysctl does not make much difference in the results.

A quick test.

<?php
$m = new Memcached();
$m->addServer('localhost', 11211);

function _log($msg) {
  print date('Y-m-d H:i:s').": {$msg}\n";
}

_log("Generating");
$data3 = array();
for ($i=0; $i< =200000; $i++) {
  $data3[md5(microtime(true) . rand(0,10100000))]=111111111111;
}
_log("Generated");
$m->setMulti($data3);
_log("Result Code from setMulti: ".$m->getResultCode());

$i=10000;
do {

  _log("Getting Slice smaller key ({$i})");
  $result = $m->getMulti(array_slice(array_keys($data3),0,$i), $cas);
  _log("Result cnt: ". count($result) . " Cas cnt:" . count($cas) . " Result code:" . $m->getResultCode());
  $i-=500;

  if ($i< =500) {
    $i+=500;
    $i-=100;
  }

  if ($i<=200) {
    $i+=100;
    $i-=10;
  }
  sleep(3);
} while($i>0);

Ubuntu: failures above 200 multiget keys

andrew@andrew-desktop:~/Downloads$

php test.php

2011-06-07 13:19:26: Generating
2011-06-07 13:19:37: Generated
2011-06-07 13:19:47: Result Code from setMulti: 0
2011-06-07 13:19:47: Getting Slice smaller key (10000)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:19:47: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:19:50: Getting Slice smaller key (9500)
2011-06-07 13:19:50: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:19:53: Getting Slice smaller key (9000)
2011-06-07 13:19:53: Result cnt: 60 Cas cnt:60 Result code:0
2011-06-07 13:19:56: Getting Slice smaller key (8500)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:19:56: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:19:59: Getting Slice smaller key (8000)
2011-06-07 13:19:59: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:20:02: Getting Slice smaller key (7500)
2011-06-07 13:20:02: Result cnt: 50 Cas cnt:50 Result code:0
2011-06-07 13:20:05: Getting Slice smaller key (7000)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:20:05: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:20:08: Getting Slice smaller key (6500)
2011-06-07 13:20:09: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:20:12: Getting Slice smaller key (6000)
2011-06-07 13:20:12: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:20:15: Getting Slice smaller key (5500)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:20:15: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:20:18: Getting Slice smaller key (5000)
2011-06-07 13:20:18: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:20:21: Getting Slice smaller key (4500)
2011-06-07 13:20:21: Result cnt: 30 Cas cnt:30 Result code:0
2011-06-07 13:20:24: Getting Slice smaller key (4000)
2011-06-07 13:20:24: Result cnt: 26 Cas cnt:26 Result code:0
2011-06-07 13:20:27: Getting Slice smaller key (3500)
2011-06-07 13:20:27: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:20:30: Getting Slice smaller key (3000)
2011-06-07 13:20:30: Result cnt: 20 Cas cnt:20 Result code:0
2011-06-07 13:20:33: Getting Slice smaller key (2500)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:20:33: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:20:36: Getting Slice smaller key (2000)
2011-06-07 13:20:36: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:20:39: Getting Slice smaller key (1500)
2011-06-07 13:20:39: Result cnt: 10 Cas cnt:10 Result code:0
2011-06-07 13:20:42: Getting Slice smaller key (1000)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:20:43: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:20:46: Getting Slice smaller key (900)
2011-06-07 13:20:46: Result cnt: 155 Cas cnt:155 Result code:0
2011-06-07 13:20:49: Getting Slice smaller key (800)
2011-06-07 13:20:49: Result cnt: 55 Cas cnt:55 Result code:0
2011-06-07 13:20:52: Getting Slice smaller key (700)
2011-06-07 13:20:52: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:20:55: Getting Slice smaller key (600)
2011-06-07 13:20:55: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:20:58: Getting Slice smaller key (500)
2011-06-07 13:20:58: Result cnt: 0 Cas cnt:0 Result code:19
2011-06-07 13:21:01: Getting Slice smaller key (400)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:21:01: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:21:04: Getting Slice smaller key (300)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:21:04: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:21:07: Getting Slice smaller key (290)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:21:07: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:21:10: Getting Slice smaller key (280)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:21:10: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:21:13: Getting Slice smaller key (270)
2011-06-07 13:21:14: Result cnt: 270 Cas cnt:270 Result code:0
2011-06-07 13:21:17: Getting Slice smaller key (260)
WATCHPOINT libmemcached/io.c:319 (memcached_io_read) We had a zero length recv()

2011-06-07 13:21:17: Result cnt: 0 Cas cnt:0 Result code:0
2011-06-07 13:21:20: Getting Slice smaller key (250)
2011-06-07 13:21:20: Result cnt: 250 Cas cnt:250 Result code:0
2011-06-07 13:21:23: Getting Slice smaller key (240)
2011-06-07 13:21:23: Result cnt: 240 Cas cnt:240 Result code:0
2011-06-07 13:21:26: Getting Slice smaller key (230)
2011-06-07 13:21:26: Result cnt: 230 Cas cnt:230 Result code:0
2011-06-07 13:21:29: Getting Slice smaller key (220)
2011-06-07 13:21:29: Result cnt: 220 Cas cnt:220 Result code:0
2011-06-07 13:21:32: Getting Slice smaller key (210)
2011-06-07 13:21:32: Result cnt: 210 Cas cnt:210 Result code:0
2011-06-07 13:21:35: Getting Slice smaller key (200)
2011-06-07 13:21:35: Result cnt: 200 Cas cnt:200 Result code:0
2011-06-07 13:21:38: Getting Slice smaller key (190)
2011-06-07 13:21:38: Result cnt: 190 Cas cnt:190 Result code:0
2011-06-07 13:21:41: Getting Slice smaller key (180)
2011-06-07 13:21:41: Result cnt: 180 Cas cnt:180 Result code:0
2011-06-07 13:21:44: Getting Slice smaller key (170)
2011-06-07 13:21:44: Result cnt: 170 Cas cnt:170 Result code:0
2011-06-07 13:21:47: Getting Slice smaller key (160)
2011-06-07 13:21:48: Result cnt: 160 Cas cnt:160 Result code:0
2011-06-07 13:21:51: Getting Slice smaller key (150)
2011-06-07 13:21:51: Result cnt: 150 Cas cnt:150 Result code:0
2011-06-07 13:21:54: Getting Slice smaller key (140)
2011-06-07 13:21:54: Result cnt: 140 Cas cnt:140 Result code:0
2011-06-07 13:21:57: Getting Slice smaller key (130)
2011-06-07 13:21:57: Result cnt: 130 Cas cnt:130 Result code:0
2011-06-07 13:22:00: Getting Slice smaller key (120)
2011-06-07 13:22:00: Result cnt: 120 Cas cnt:120 Result code:0
2011-06-07 13:22:03: Getting Slice smaller key (110)
2011-06-07 13:22:03: Result cnt: 110 Cas cnt:110 Result code:0
2011-06-07 13:22:06: Getting Slice smaller key (100)
2011-06-07 13:22:06: Result cnt: 100 Cas cnt:100 Result code:0
2011-06-07 13:22:09: Getting Slice smaller key (90)
2011-06-07 13:22:09: Result cnt: 90 Cas cnt:90 Result code:0
2011-06-07 13:22:12: Getting Slice smaller key (80)
2011-06-07 13:22:12: Result cnt: 80 Cas cnt:80 Result code:0
2011-06-07 13:22:15: Getting Slice smaller key (70)
2011-06-07 13:22:16: Result cnt: 70 Cas cnt:70 Result code:0
2011-06-07 13:22:19: Getting Slice smaller key (60)
2011-06-07 13:22:19: Result cnt: 60 Cas cnt:60 Result code:0
2011-06-07 13:22:22: Getting Slice smaller key (50)
2011-06-07 13:22:22: Result cnt: 50 Cas cnt:50 Result code:0
2011-06-07 13:22:25: Getting Slice smaller key (40)
2011-06-07 13:22:25: Result cnt: 40 Cas cnt:40 Result code:0
2011-06-07 13:22:28: Getting Slice smaller key (30)
2011-06-07 13:22:28: Result cnt: 30 Cas cnt:30 Result code:0
2011-06-07 13:22:31: Getting Slice smaller key (20)
2011-06-07 13:22:31: Result cnt: 20 Cas cnt:20 Result code:0
2011-06-07 13:22:34: Getting Slice smaller key (10)
2011-06-07 13:22:34: Result cnt: 10 Cas cnt:10 Result code:0
 

Debian: no failures

andrew@bb1:~$ php test.php
2011-06-07 13:20:38: Generating
2011-06-07 13:20:41: Generated
2011-06-07 13:20:52: Result Code from setMulti: 0
2011-06-07 13:20:52: Getting Slice smaller key (10000)
2011-06-07 13:20:53: Result cnt: 10000 Cas cnt:10000 Result code:0
2011-06-07 13:20:56: Getting Slice smaller key (9500)
2011-06-07 13:20:56: Result cnt: 9500 Cas cnt:9500 Result code:0
2011-06-07 13:20:59: Getting Slice smaller key (9000)
2011-06-07 13:20:59: Result cnt: 9000 Cas cnt:9000 Result code:0
2011-06-07 13:21:02: Getting Slice smaller key (8500)
2011-06-07 13:21:02: Result cnt: 8500 Cas cnt:8500 Result code:0
2011-06-07 13:21:05: Getting Slice smaller key (8000)
2011-06-07 13:21:05: Result cnt: 8000 Cas cnt:8000 Result code:0
2011-06-07 13:21:08: Getting Slice smaller key (7500)
2011-06-07 13:21:09: Result cnt: 7500 Cas cnt:7500 Result code:0
2011-06-07 13:21:12: Getting Slice smaller key (7000)
2011-06-07 13:21:12: Result cnt: 7000 Cas cnt:7000 Result code:0
2011-06-07 13:21:15: Getting Slice smaller key (6500)
2011-06-07 13:21:15: Result cnt: 6500 Cas cnt:6500 Result code:0
2011-06-07 13:21:18: Getting Slice smaller key (6000)
2011-06-07 13:21:18: Result cnt: 6000 Cas cnt:6000 Result code:0
2011-06-07 13:21:21: Getting Slice smaller key (5500)
2011-06-07 13:21:22: Result cnt: 5500 Cas cnt:5500 Result code:0
2011-06-07 13:21:25: Getting Slice smaller key (5000)
2011-06-07 13:21:25: Result cnt: 5000 Cas cnt:5000 Result code:0
2011-06-07 13:21:28: Getting Slice smaller key (4500)
2011-06-07 13:21:28: Result cnt: 4500 Cas cnt:4500 Result code:0
2011-06-07 13:21:31: Getting Slice smaller key (4000)
2011-06-07 13:21:31: Result cnt: 4000 Cas cnt:4000 Result code:0
2011-06-07 13:21:34: Getting Slice smaller key (3500)
2011-06-07 13:21:34: Result cnt: 3500 Cas cnt:3500 Result code:0
2011-06-07 13:21:37: Getting Slice smaller key (3000)
2011-06-07 13:21:38: Result cnt: 3000 Cas cnt:3000 Result code:0
2011-06-07 13:21:41: Getting Slice smaller key (2500)
2011-06-07 13:21:41: Result cnt: 2500 Cas cnt:2500 Result code:0
2011-06-07 13:21:44: Getting Slice smaller key (2000)
2011-06-07 13:21:44: Result cnt: 2000 Cas cnt:2000 Result code:0
2011-06-07 13:21:47: Getting Slice smaller key (1500)
2011-06-07 13:21:47: Result cnt: 1500 Cas cnt:1500 Result code:0
2011-06-07 13:21:50: Getting Slice smaller key (1000)
2011-06-07 13:21:50: Result cnt: 1000 Cas cnt:1000 Result code:0
2011-06-07 13:21:53: Getting Slice smaller key (900)
2011-06-07 13:21:54: Result cnt: 900 Cas cnt:900 Result code:0
2011-06-07 13:21:57: Getting Slice smaller key (800)
2011-06-07 13:21:57: Result cnt: 800 Cas cnt:800 Result code:0
2011-06-07 13:22:00: Getting Slice smaller key (700)
2011-06-07 13:22:00: Result cnt: 700 Cas cnt:700 Result code:0
2011-06-07 13:22:03: Getting Slice smaller key (600)
2011-06-07 13:22:03: Result cnt: 600 Cas cnt:600 Result code:0
2011-06-07 13:22:06: Getting Slice smaller key (500)
2011-06-07 13:22:06: Result cnt: 500 Cas cnt:500 Result code:0
2011-06-07 13:22:09: Getting Slice smaller key (400)
2011-06-07 13:22:10: Result cnt: 400 Cas cnt:400 Result code:0
2011-06-07 13:22:13: Getting Slice smaller key (300)
2011-06-07 13:22:13: Result cnt: 300 Cas cnt:300 Result code:0
2011-06-07 13:22:16: Getting Slice smaller key (290)
2011-06-07 13:22:16: Result cnt: 290 Cas cnt:290 Result code:0
2011-06-07 13:22:19: Getting Slice smaller key (280)
2011-06-07 13:22:19: Result cnt: 280 Cas cnt:280 Result code:0
2011-06-07 13:22:22: Getting Slice smaller key (270)
2011-06-07 13:22:22: Result cnt: 270 Cas cnt:270 Result code:0
2011-06-07 13:22:25: Getting Slice smaller key (260)
2011-06-07 13:22:26: Result cnt: 260 Cas cnt:260 Result code:0
2011-06-07 13:22:29: Getting Slice smaller key (250)
2011-06-07 13:22:29: Result cnt: 250 Cas cnt:250 Result code:0
2011-06-07 13:22:32: Getting Slice smaller key (240)
2011-06-07 13:22:32: Result cnt: 240 Cas cnt:240 Result code:0
2011-06-07 13:22:35: Getting Slice smaller key (230)
2011-06-07 13:22:35: Result cnt: 230 Cas cnt:230 Result code:0
2011-06-07 13:22:38: Getting Slice smaller key (220)
2011-06-07 13:22:38: Result cnt: 220 Cas cnt:220 Result code:0
2011-06-07 13:22:41: Getting Slice smaller key (210)
2011-06-07 13:22:41: Result cnt: 210 Cas cnt:210 Result code:0
2011-06-07 13:22:44: Getting Slice smaller key (200)
2011-06-07 13:22:44: Result cnt: 200 Cas cnt:200 Result code:0
2011-06-07 13:22:47: Getting Slice smaller key (190)
2011-06-07 13:22:48: Result cnt: 190 Cas cnt:190 Result code:0
2011-06-07 13:22:51: Getting Slice smaller key (180)
2011-06-07 13:22:51: Result cnt: 180 Cas cnt:180 Result code:0
2011-06-07 13:22:54: Getting Slice smaller key (170)
2011-06-07 13:22:54: Result cnt: 170 Cas cnt:170 Result code:0
2011-06-07 13:22:57: Getting Slice smaller key (160)
2011-06-07 13:22:57: Result cnt: 160 Cas cnt:160 Result code:0
2011-06-07 13:23:00: Getting Slice smaller key (150)
2011-06-07 13:23:00: Result cnt: 150 Cas cnt:150 Result code:0
2011-06-07 13:23:03: Getting Slice smaller key (140)
2011-06-07 13:23:03: Result cnt: 140 Cas cnt:140 Result code:0
2011-06-07 13:23:06: Getting Slice smaller key (130)
2011-06-07 13:23:07: Result cnt: 130 Cas cnt:130 Result code:0
2011-06-07 13:23:10: Getting Slice smaller key (120)
2011-06-07 13:23:10: Result cnt: 120 Cas cnt:120 Result code:0
2011-06-07 13:23:13: Getting Slice smaller key (110)
2011-06-07 13:23:13: Result cnt: 110 Cas cnt:110 Result code:0
2011-06-07 13:23:16: Getting Slice smaller key (100)
2011-06-07 13:23:16: Result cnt: 100 Cas cnt:100 Result code:0
2011-06-07 13:23:19: Getting Slice smaller key (90)
2011-06-07 13:23:19: Result cnt: 90 Cas cnt:90 Result code:0
2011-06-07 13:23:22: Getting Slice smaller key (80)
2011-06-07 13:23:22: Result cnt: 80 Cas cnt:80 Result code:0
2011-06-07 13:23:25: Getting Slice smaller key (70)
2011-06-07 13:23:25: Result cnt: 70 Cas cnt:70 Result code:0
2011-06-07 13:23:28: Getting Slice smaller key (60)
2011-06-07 13:23:29: Result cnt: 60 Cas cnt:60 Result code:0
2011-06-07 13:23:32: Getting Slice smaller key (50)
2011-06-07 13:23:32: Result cnt: 50 Cas cnt:50 Result code:0
2011-06-07 13:23:35: Getting Slice smaller key (40)
2011-06-07 13:23:35: Result cnt: 40 Cas cnt:40 Result code:0
2011-06-07 13:23:38: Getting Slice smaller key (30)
2011-06-07 13:23:38: Result cnt: 30 Cas cnt:30 Result code:0
2011-06-07 13:23:41: Getting Slice smaller key (20)
2011-06-07 13:23:41: Result cnt: 20 Cas cnt:20 Result code:0
2011-06-07 13:23:44: Getting Slice smaller key (10)
2011-06-07 13:23:44: Result cnt: 10 Cas cnt:10 Result code:0

Share

2 Responses to Memcached multigets – ubuntu vs debian

Avatar

Tomasz Sh4dow Budzyński

June 15th, 2011 at 7:35 am

I think You should check your Ubuntu. On my Kubuntu i’m not getting any errors and all work fine.
Kubuntu 11.04
PHP 5.3.5-1ubuntu7.2 with Suhosin-Patch (cli) (built: May 2 2011 23:00:17)

Avatar

Andrew Johnstone’s Blog: Memcached multigets – ubuntu vs debian | PHP Boutique

June 15th, 2011 at 7:39 am

[...] er memcached multigets mit einer großen Anzahl von Schlüsseln verwenden wollte – es war werfen Ausfälle , aber nur wenn er es versucht auf Ubuntu. Ich verbrachte eine Weile gestern erforschen, warum [...]

Comment Form

About this blog

Andrew Johnstone is a software engineer / lead developer working at Everlution Software.

Photostream

  • Multiple Ucarp on the same host | PaoloBertasi: […] http://ajohnstone.com/achives/running-several-vips-on-the-same-interface-with-ucarp-and-ha [...]
  • mohit: I am working the same on windows server 2008 and mysql version 5.1.39 ,but it is not working and i a [...]
  • Steve: Hi there... sorry that this is old, but I'm trying to use your script to check for my usage per seco [...]
  • vinodh: great. thanks . My boss wanted me to configure multiple mysql instances on same physical machine . i [...]
  • andrew.johnstone: In the example above this was in fact using spl_autoload_register. I've never debugged this properly [...]

GitHub

GitHub Octocat

ajohnstone @ GitHub

  • Status updating...