News Feed
Sections




News Archive
Looking for more information on how to do PHP the right way? Check out PHP: The Right Way

Developer Tutorials Blog:
Parallel web scraping in PHP cURL multi functions
July 29, 2008 @ 07:57:00

The Developer Tutorials blog has posted a tutorial about scraping other website information in parallel (with their permission, of course) with the help of the cURL extension.

For anyone who's ever tried to fetch multiple resources over HTTP in PHP, the logic is trivial, but one key challenge is ever-present: latency delays. While web servers have perfectly good downstream links, latencies can increase script execution time tenfold just by downloading a few external URLs. But there's a simple solution: parallel cURL operations. In this tutorial, I'll show you how to use the "multi" functions in PHP's cURL library to get around this quickly and easily.

He starts with a basic cURL example, grabbing the content from example.com and putting it into a variable. He modifies this to make it a bit more complex and to run multiple fetches in parallel - creating more than one cURL object and using the culr_multi_* methods to manage them.

0 comments voice your opinion now!
webscraping curl function multi parallel tutorial


blog comments powered by Disqus

Similar Posts

Norbert Mocsnik's Blog: Setting Up DokuWiki with Simple Authentication

Vance Lucas' Blog: Get Only Public Class Properties for the Current Class in PHP

WeberDev.com: User Authentication With patUser (Part 1)

PHPBuilder.com: Using the ADOdb library with Oracle and PHP

ProDevTips.com: Writing a CMS/Community with Smarty and the Zend Framework: Part 6


Community Events

Don't see your event here?
Let us know!


interview conference opinion api example list voicesoftheelephpant library performance series php7 release version community podcast symfony2 laravel extension introduction framework

All content copyright, 2015 PHPDeveloper.org :: info@phpdeveloper.org - Powered by the Solar PHP Framework