Joe Watkins have just announced the availability of Parallel 1.0.0 for PHP, the succinct parallel concurrency API for PHP7. Now you can easily write a web crawler in PHP or do any kind of concurrency without leaving PHP. A very interesting page in the documentation explain the philosophy behind parallel, with some details about the internal implementation of the code : Do not communicate by sharing memory; instead, share memory by communicating and Data should have a definitive single owner.
A simple Hello world in parallel become :
<?php $runtime = new \parallel\Runtime(); $future = $runtime->run(function(){ for ($i = 0; $i < 500; $i++) echo "*"; return "easy"; }); for ($i = 0; $i < 500; $i++) { echo "."; } printf("\nUsing \\parallel\\Runtime is %s\n", $future->value());
This may output something like (truncated):
.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*. Using \parallel\Runtime is easy
A more complete and advanced example of web crawler showing in couple of hundred lines : Futures, Channels (buffered, unbuffered, synchros), Events using parallel producer/consumer pattern, is available in this gist.
More information about installation and API are available here : https://php.net/parallel