kelatebaru

using column stores instead of row based stores can reduce access logs from 10 GB to 130 MB of disk space reading compressed log files is 4 times faster than reading uncompressed files from hard disk column stores can speed up analytical queries by a factor of 18-58 Normally, log files from a web server are stored in a single file. For archiving, log files get compressed with gzip. A typical line in a log file represents one request and looks like this: 173.15.3.XXX - - [30/May/2012:00:37:35 +0200] "GET /cms/ext/files/Sgs01Thumbs/sgs_pmwiki2.jpg HTTP/1.1" 200 14241 "http://www.simple-groupware.de/cms/ManualPrint" "Mozilla/5.0 (Windows NT 5.1; rv:12.0) Gecko/20100101 Firefox/12.0" Compression speeds up reading the log file: $start = microtime(true); $fp = gzopen("httpd.log.gz", "r"); while (!gzeof($fp)) gzread($fp, 8192); gzclose($fp); echo (microtime(true)-$start)."s\n"; // 26s $start = microtime(true); $fp = fopen(...

To determine the maximum capacity of a web page, Apache ab is often used in the first step. Fetching one URL very often is optimal for caching and gives a best case . To get the worst case for caching, it is necessary to fetch different URLs in a random order. Here is a PHP script to walk randomly on a web page: To get the average case concerning caching and response times, we need to choose the most relevant links. For example, we skip links from headers and footers. This can be done by using a different xpath expression in the code: // fetch all links under <div id="content">...</div> $xpath = '//div[@id="content"]//a'; // fetch all links under <div id="content"> and <div id="menu"> $xpath = '//div[@id="content" or @id="menu"]//a'; To make the benchmark more realistic, you can define a waiting period between two requests: Uncomment "// sleep(1)" at the end of the scrip...