PHP caching: shm vs. apc vs. memcache vs. mysql vs. file cache (update: fill apc from cron)

Lessons learned:
  • shm/apc are 32-60 times faster than memcached or mysql
  • shm/apc are 2 times faster than php file cache with apc
  • php file cache with apc is 15-24 times faster than memcached or mysql
  • mysql is 2 times faster than memcached when storing more than 400 bytes
  • memcached is 2 times faster than mysql when storing less than 400 bytes
  • php file cache with apc is 2-3 times faster than normal file cache
  • php file cache without apc is 8 times slower than normal file cache

Tests were made with PHP 5.3.10, MySQL 5.5.29, memcached 1.4.13, 64bit, 3.4GHz (QEMU):


shm
0.031 0.020 0.021 0.021 0.026 0.028 0.032 0.042 0.084 0.155 0.290 0.629 0.110
Total: 1.489, Avg: 0.115

apc
0.025 0.025 0.025 0.026 0.031 0.036 0.043 0.060 0.106 0.171 0.328 0.756 0.097
Total: 1.728, Avg: 0.133

memcache
3.116 3.014 3.005 3.072 3.077 3.910 3.929 4.067 4.308 10.371 15.323 25.013 3.281
Total: 85.488, Avg: 6.576

memcache socket
1.736 1.756 1.981 1.780 1.809 1.907 1.941 1.983 2.225 9.368 14.071 24.897 1.979
Total: 67.435, Avg: 5.187

memcached
2.241 2.540 2.713 2.769 2.897 3.249 4.286 5.298 7.729 10.539 16.021 28.060 2.578
Total: 90.919, Avg: 6.994

mysql myisam
3.267 3.291 3.310 3.295 3.700 3.777 3.888 4.078 4.368 6.272 6.930 9.626 3.726
Total: 59.529, Avg: 4.579

mysql memory
3.238 3.360 3.470 3.502 3.310 3.346 3.681 4.108 4.370 6.286 7.279 7.079 3.397
Total: 56.426, Avg: 4.340

file cache
0.593 0.595 0.593 0.609 0.546 0.563 0.574 0.600 0.648 0.800 1.115 1.956 0.775
Total: 9.966, Avg: 0.767

php file cache
0.177 0.176 0.175 0.180 0.187 0.188 0.195 0.210 0.236 0.318 0.479 0.901 0.228
Total: 3.650, Avg: 0.281

(10b  0.1k  0.3k  0.5k  1k    2k    4k    8k    16k   32k   64k   128k  array)
Notes: Numbers in seconds, smaller numbers are better, connection times for memcached and mysql are not counted, file cache was done on tmpfs.

Cold cache, fill apc cache from cron:

All entries in apc cache are kept in shared memory. Shared memory is available to the web server and is not shared with the command line php (php-cli). When the web server is restarted, the cache is empty and refilling the complete cache can create performance problems. To fill the apc cache with a cron job, we use a second script to dump the cache to a binary file and load it inside the web server: 


// php cron.php
$data = array("cached"=>1, "hello"=>"world");
apc_store($data);
apc_bin_dumpfile(array(), array_keys($data), "/var/cache/apc_cron.bin");

// bootstrap.php, http://myserver.com/...
if (apc_fetch("cached")===false) apc_bin_loadfile("/var/cache/apc_cron.bin");

echo apc_fetch("hello"); // gives "world"

Related articles:

Here is the test script:


<?php
// memcached -m 64 -s /tmp/m.sock -a 0777 -p 0 -u memcache
// memcached -m 64 -l 127.0.0.1 -p 11211 -u memcache
set_time_limit(3600);
error_reporting(E_ALL);
ini_set("display_errors", 1);
ini_set("apc.enable_cli", 1);
mysqli_report(MYSQLI_REPORT_STRICT | MYSQLI_REPORT_ERROR);

$data = array( // test are range from 10-128,000 bytes
  "1111111110",
  str_repeat("1111111110", 10),
  str_repeat("1111111110", 30),
  str_repeat("1111111110", 50),
  str_repeat("1111111110", 100),
  str_repeat("1111111110", 200),
  str_repeat("1111111110", 400),
  str_repeat("1111111110", 800),
  str_repeat("1111111110", 1600),
  str_repeat("1111111110", 3200),
  str_repeat("1111111110", 6400),
  str_repeat("1111111110", 12800),
  array(0=>"1111111110", "id2"=>"hello world", "id3"=>"foo bar", "id4"=>42)
);

echo "shm\n";
$t = array();
foreach ($data as $key=>$val) $t[] = shm(4000+$key, $val);
echo stats($t);

echo "apc\n";
$t = array();
foreach ($data as $key=>$val) $t[] = apc((string)$key, $val);
echo stats($t);

echo "memcache\n";
$t = array();
$m = memcache_connect("127.0.0.1", 11211);
foreach ($data as $key=>$val) $t[] = memcache($m, (string)$key, $val);
echo stats($t);

echo "memcache socket\n";
$t = array();
$m = memcache_connect("unix:///tmp/m.sock", 0);
foreach ($data as $key=>$val) $t[] = memcache($m, (string)$key, $val);
echo stats($t);

echo "memcached\n";
$t = array();
$m = new Memcached();
$m->addServer("127.0.0.1", 11211);
foreach ($data as $key=>$val) $t[] = memcached($m, (string)$key, $val);
echo stats($t);

/* not in memcached 1.x
echo "memcached socket\n";
$t = array();
$m = new Memcached();
$m->addServer("unix:///tmp/m.sock", 0);
foreach ($data as $key=>$val) $t[] = memcached($m, (string)$key, $val);
echo stats($t);
*/

echo "mysql myisam\n";
$t = array();
$m = new mysqli("127.0.0.1", "root", "", "t1");
mysqli_query($m, "drop table if exists t1.cache");
mysqli_query($m, "create table t1.cache (id int primary key, data mediumtext) engine=myisam");
foreach ($data as $key=>$val) $t[] = mysql_cache($m, $key, $val);
echo stats($t);

echo "mysql memory\n";
$t = array();
mysqli_query($m, "drop table if exists t1.cache");
mysqli_query($m, "create table t1.cache (id int primary key, data varchar(65500)) engine=memory");
foreach ($data as $key=>$val) $t[] = mysql_cache($m, $key, $val);
echo stats($t);

echo "file cache\n";
$t = array();
foreach ($data as $key=>$val) $t[] = file_cache((string)$key, $val);
echo stats($t);

echo "php file cache\n";
$t = array();
foreach ($data as $key=>$val) $t[] = php_cache((string)$key, $val);
echo stats($t);

function stats($t) {
  return "\nTotal: ".number_format(array_sum($t), 3).", ".
    "Avg: ".number_format(array_sum($t) / count($t), 3)."\n\n";
}

function format($num) {
  return number_format($num, 3);
}

function shm($id, $data) {
  if (is_array($data)) {
    $arr = true;
    $data = serialize($data);
  } else $arr = false;
  $len = strlen($data);
  $shm_id = shmop_open($id, "c", 0644, $len);
  shmop_write($shm_id, $data, 0);
  $start = microtime(true);
  if ($arr) {
    for ($i=0; $i<100000; $i++) $v = unserialize(shmop_read($shm_id, 0, $len));
  } else {
    for ($i=0; $i<100000; $i++) $v = shmop_read($shm_id, 0, $len);
  }
  echo format($end = microtime(true)-$start)." ";
  shmop_close($shm_id);
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function apc($id, $data) {
  apc_store($id, $data);
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) $v = apc_fetch($id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function memcache($m, $id, $data) {
  memcache_set($m, $id, $data);
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) $v = memcache_get($m, $id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function memcached($m, $id, $data) {
  $m->set($id, $data);
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) $v = $m->get($id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function mysql_cache($m, $id, $data) {
  $d = is_array($data) ? serialize($data) : $data;
  mysqli_query($m, "insert into t1.cache values (".$id.", '".$d."')");
  $start = microtime(true);
  if (is_array($data)) {
    for ($i=0; $i<100000; $i++) {
      $v = mysqli_query($m, "SELECT data FROM t1.cache WHERE id=".$id)->fetch_row();
      $v = unserialize($v[0]);
    }
  } else {
    for ($i=0; $i<100000; $i++) {
      $v = mysqli_query($m, "SELECT data FROM t1.cache WHERE id=".$id)->fetch_row();
    }
  }
  echo format($end = microtime(true)-$start)." ";
  assert(substr($v[0], 0, 10)=="1111111110");
  return $end;
}

function file_cache($id, $data) {
  file_put_contents($id, is_array($data) ? serialize($data) : $data);
  $start = microtime(true);
  if (is_array($data)) {
    for ($i=0; $i<100000; $i++) $v = unserialize(file_get_contents($id));
  } else {
    for ($i=0; $i<100000; $i++) $v = file_get_contents($id);
  }
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function php_cache($id, $data) {
  $id .= ".php";
  $data = is_array($data) ? var_export($data, 1) : "'".$data."'";
  file_put_contents($id, "<?php\n\$v=".$data.";");
  touch($id, time()-10); // needed for APC's file update protection
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) include($id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

Comments

Post a Comment

Popular posts from this blog

How to show only month and year fields in android Date-picker?

Image
Android default DatePicker show day/month/year fields but you have to customize this DatePicker as your requirements. Suppose that you want to show only month and year fields in android datepicker . To do this, you have to customize the Dialog DatePicker source code but it is not an easy task. In this tutorial I will show you a tricky way to solve this problem. Lets see how to show only month and year field in android DatePicker as a dialog.   Step to show only month and year fields in android date-picker: Some days ago I got this same problem and here I am trying to explain this in a sequential manner. I want to show a dialog datepicker when user click on an EditText. So if user click on EditText, A dialog datepicker will be invocked and it will show only month and year fields. To do this, at first we have to create a xml layout to hold the EditText fields. So create an android project and name it DatePickerExample. In the main.xml file just try to write this simple code....
Read more

How to construct a B+ tree with example

Image
A B+ tree contains n-1 search key values K1K2…….Kn-1 and have n pointers.   In a B+ tree the data are ordered  sequentially.  The leaf node may hold up to n pointers and must hold at least N/2 pointers. The root node must hold at least two pointers, unless the tree consists of only one node.  Now we construct a B+ tree. Construct a B+ tree for the following set of key values: (2,3,5,7,11,17,19,23,29,31) Assume that the tree is initially empty and values are added in ascending order. Construct a B+ tree where the pointer number is Four. Solution: When a node exceeds n-1 search key value then we split it into two nodes. First node contains (ceiling(n-1)/2) values.  2 nd node contains remaining node. Copy the smallest search key value of the second node to the parent node. Let's start, we know each node has N pointer and has N-1 search key values. In our example pointer number is 4, so the search key value is 4-1=3. So each node has 4 pointers and 3 search key val...
Read more

Conflict Serializability in database

Image
To maintains database state consistency we introduce serializability. Using serializability we can understand which schedule will ensure consistency, and which schedule will not. When we are capable of converting a concurrent schedule into a serial schedule then we cal it serializable schedule. A schedule is serializable if it is equivalent to a serial schedule. We divided this concepts into two parts 1. Conflict serializability  & 2. View serializability In this discussion I just clarify you the conflict serializability. Consider two instruction read, write and the data item A and B. When two instruction execute different data item such as read(A) and read(B), then we can swap the instruction without affecting the result of the schedule. But when the data item is equivalent to one another we need to consider following four condition.    a.   read (A ),    read ( A ).   they   don’t conflict.    b.   read ( Q),  ...
Read more