News Feed
Sections




News Archive
Looking for more information on how to do PHP the right way? Check out PHP: The Right Way

PHPBuilder.com:
Build a PHP Link Scraper with cURL
January 15, 2010 @ 10:02:45

On PHPBuilder.com today there's a new tutorial posted about building a link scraping script with the combination of PHP and cURL (the script pulls in a page, grabs all of the links off of it and follows them, etc).

I actually built this a few years ago because I had grandiose visions of becoming the next Google. Clearly, that did not happen, mostly because my localhost, database, and bandwidth are not infinite. Yet this little robot has quite interesting applications and uses if you really have the time to play with and fine-tune it.

You'll need to have cURL support built into your PHP installation to get the scripts working, but the actual code itself is pretty simple. Curl and XPath do most of the heavy lifting of finding and following the links and its easy enough to drop them into a MySQL table from there. You can download the source here.

1 comment voice your opinion now!
link scraper curl xpath mysql tutorial


blog comments powered by Disqus

Similar Posts

Job Posting: Ubersmith Seeks PHP/MySQL Developers (Troy, NY)

Evert Pot's Blog: Creating a Gopher server with PHP and InetD

Developer.com: Forms Validation with Symfony and Prototype

Dennis Chung's Blog: Server Core + IIS7 + PHP + MySQL (and Wordpress)

Kevin Schroeder: If you develop for Magento, know your indexes


Community Events

Don't see your event here?
Let us know!


example release laravel laravel5 php7 framework podcast voicesoftheelephpant interview library extension video security version community introduction api opinion series language

All content copyright, 2015 PHPDeveloper.org :: info@phpdeveloper.org - Powered by the Solar PHP Framework