Looking for more information on how to do PHP the right way? Check out PHP: The Right Way

Grok Interactive:
Importing Large CSV Files with PHP Part 1: Import Using One Query
Sep 23, 2015 @ 12:19:33

The Grok Interactive blog has posted a tutorial, the first part in a series, showing you how to work with large CSV files in PHP.

Importing CSV files into your application can become a problem when the file is really big, > 65,000 rows big. Each row of the file needs to be parsed, converted into an object, and then saved to a database. All of this needs to happen within a 30 second timeout window. It may sound like an impossible task, but there are actually a couple of solutions that can solve this problem. While working on a project at Grok, I was tasked with doing exactly that.

He talks about the method he tried initially for parsing the large files, splitting it up into different files and processing them as chunks. He points out that it relies on the file system, though, and this made it difficult to debug. He finally came up with a different, more simple solution: importing the files directly into MySQL via a LOAD DATA LOCAL INFILE command. He shows how to set this up in a controller and "importer" class that handles the upload and import via the importFileContents method (complete code included). He walks through what the code is doing and includes a few notes about the configuration of the database connection to specify additional options on the PDO connection to allow the local file load.

tagged: tutorial csv file import large processing chunk mysql load file query

Link: http://www.grok-interactive.com/blog/import-large-csv-into-mysql-with-php-part-1/

PHPUnit 4.7 and Three Shades of Green
Jun 08, 2015 @ 12:57:25

Sebastian Bergmann has posted a guide to PHPUnit 4.7 and some of the changes/new features it introduces.

PHPUnit 4.7 introduces a couple of small improvements. For instance, PHPUnit's PHPT test runner now supports --INI-- sections, information about the PHP runtime used is now printed in verbose mode, and a warning is now printed when code coverage data is collected but no whitelist is configured.

He also talks about the support that's been added improving the output of the HTML version of the code coverage reports, showing different colors based on how well covered the lines are. He also briefly looks ahead to PHPUnit 5, the versions it will support and the plans for release.

tagged: phpunit unittest v47 small medium large coverage shade phpunit5

Link: https://thephp.cc/news/2015/06/phpunit-4-7-and-three-shades-of-green

SitePoint PHP Blog:
How to Encrypt Large Messages with Asymmetric Keys and phpseclib
Jan 20, 2015 @ 11:40:51

On the SitePoint PHP blog today David Brumbaugh shows you how to encrypt large messages with phpseclib and asymmetric keys. phpseclib is a PHP library specifically designed to handle encryption and decryption in an easy-to-use way.

Most of us understand the need to encrypt sensitive data before transmitting it. Encryption is the process of translating plaintext (i.e. normal data) into ciphertext (i.e. secret data). During encryption, plaintext information is translated to ciphertext using a key and an algorithm. To read the data, the ciphertext must be decrypted (i.e. translated back to plaintext) using a key and an algorithm. [...] A core problem to be solved with any encryption algorithm is key distribution. How do you transmit keys to those who need them in order to establish secure communication? The solution to the problem depends on the nature of the keys and algorithms.

He talks some about the difference between symmetric and asymmetric algorithms and some advice about the selection of the right one (or ones) to use in your app. He also talks briefly about the problem with RSA keys, mostly that it has limits on the amount of text it can encrypt. His solution is to "encrypt the message with a symmetric key, then asymmetrically encrypt the key and attach it to the message". He explains the encryption/decryption process step by step and starts in showing the code to make phpseclib do the work. He shows how to generate the keys, build the encrypt function and the decrypt function with about 30 lines of code each.

tagged: encrypt decrypt large message asymetric key phpseclib tutorial

Link: http://www.sitepoint.com/encrypt-large-messages-asymmetric-keys-phpseclib/

Christian Weiske:
PHP 5.6: Large file upload support
Dec 11, 2013 @ 11:09:47

Christian Weiske has posted information about a feature in the upcoming PHP 5.6 version of the language - large file upload support. This new feature allows files over 4GB to be uploaded correctly.

PHP version 5.6 brings support for file uploads larger than 2GiB. We can say "thank you" to Ralf Lang for the initial patch that fixes bug #44522 , which was open since 2008. During testing uploads of files with a size of 4 - 11GiB on my PHP-CGI setup, I noticed that files above 4GiB did not get uploaded correctly. Michael Wallner was quick to fix that bug, and now 5.6 has fully working support for big files.

PHP 5.6 is still in development and some other new features are slated to be added to it. You can find some of them listed in the RFC section of the PHP wiki.

tagged: php56 large file upload bug patch

Link: http://cweiske.de/tagebuch/php-large-file-uploads.htm

Ilia Alshanetsky's Blog:
Performance Analysis of isset() vs array_key_exists()
Mar 06, 2012 @ 08:44:31

Ilia Alshanetsky has posted about a performance difference he's found between using the isset and array_key_exists functions in PHP to see if a value exists.

At Confoo I had an interesting conversation with Guilherme Blanco regarding the fact that in Doctrine 2 they had a performance issue due to usage of array_key_exists() and how it was significantly slower than isset(). His anecdotal example was that doing isset() took 0.5 seconds, while array_key_exists() for the same operation took 5 seconds! That seemed wrong [...] so, I've decided to do a quick benchmark using a 5,000 element array.

His benchmarking code is included - it just loads up a simple data set from a file of "words" and measures the microtime between the isset and array_key_exists calls. His results do show that isset is the faster of the two (by 2.5x) but it's still a super small micro-optimization that won't gain you much in the end.

The bottom line is that if your application does not need to distinguish between an array key that does not exist and one whose value happens to be NULL you should use isset() because it happens to be a little faster.
tagged: isset arraykeyexists benchmark large array code


Kevin Schroeder's Blog:
fatal: The remote end hung up unexpectedly
Nov 04, 2011 @ 12:55:28

Kevin Schroeder has a quick tip for anyone using phpcloud.com and having trouble with git and "remote end hung up" error messages.

If you are using phpcloud.com and are experiencing errors with git [...] and you are trying to push large files (not sure what is defined as "large") you may need to change some git settings.

He points out two settings - one for Windows and the other for Linux - that increase the buffer size to handle larger files that might be included in your repository.

tagged: phpcloud git problem large file buffer size


Martin Sikora's Blog:
Storing arrays using JSON, serialize and var_export
Aug 09, 2011 @ 09:31:51

Martin Sikora was working on an application that used a large dataset (in an array) and found some interesting things in regards to PHP's resulting loading time and saving time in four different types of arrays.

Recently I was dealing with precessing and storing large arrays in PHP (around 100 000 items) and I found out some quiet surprising facts that are very useful in performance critical applications. [...] When I started looking for some benchmark I found article Cache a large array: JSON, serialize or var_export?. That is really good but I wanted to compare a few more things.

He tested with four different array types including associative with an integer value and numeric index with a string value at sizes of 10, 100, 1,000 and 10,000 items. He ran his tests with the json methods, serializing them and a var_export. There's graphs of his results for each included in the post with some interesting variations between the different array types.

tagged: json serialize varexport large array dataset benchmark


Community News:
ElePHPants - the Next generation
Dec 17, 2010 @ 06:52:22

If you've been trying to get your hands on one of the cuddly little mascots for PHP (the elePHPant) but haven't managed to yet, there's some good news! Another run of the fuzzy little animals is being done (the 6th) and this time they're offering something new - pink elePHPants.

As we are now out of every of the 5 first generation of the elephpants, it is time to start a 6th. You'll find here all information to include yourself. [...] Fill in your elephpants wishes. We do not need any payement now. We will contact you directly before starting the generation for the actual payement.

Their schedule hopes to end the pre-order process on December 20th (just three days away) and to strat production on these pre-orders by January 10th. The end results would be shipped out for delivery in April 2011. If you'd like to lay claim to some of your own, go over to the order form and select the size (large/small) and the color (blue/pink) and fill in the contact info. Unfortunately, because of production restrictions, single elePHPants cannot be ordered, so consider getting together with a local user group and all chipping in for a box!

tagged: elephpant order preorder pink blue large small animal


SitePoint PHP Blog:
How to Upload Large Files in PHP
Aug 17, 2010 @ 08:44:18

On the SitePoint PHP blog today Craig Buckler talks about uploading large files in your PHP application. He points to two other resources - this manual page and this introductory tutorial about handling file uploads to get the ball rolling.

One of the most popular uses is image uploads. Your users can submit photographs from a form without resorting to FTP or other convoluted methods. HTML5 and Flash also permit drag and drop, so the operation is likely to become easier as browsers evolve. This is where the problems can begin.

He points out the large size of the images most modern cameras work with and how PHP, with its basic settings, can't handle a lot of the resulting images. He mentions the upload_max_filesize and post_max_size settings you can set in either your php.ini or via an .htaccess (or even in your script). There's also a few helpful comments with more tips on large file handling.

tagged: upload large file tutorial phpini setting


Programming Facts Blog:
Upload large(Big) files in PHP using .htaccess
Feb 17, 2010 @ 13:19:06

Rakshit Patel has posted a tip to the Programming Facts blog that those out there wanting to upload larger files through your application - change your settings via one of three ways to tell PHP it's okay.

I have seen many developers who find difficulties when working with larger files upload in php. When files which are too large in size [...] If you are uploading file which is larger than 2MB size than here i am showing you the way to upload larger files using PHP.

The method's pretty much the same in each of the three methods. You can either have the settings in your httpd.conf (if you have access to it), in the php.ini or in a .htaccess file in the directory your PHP script is in.

tagged: upload large file tutorial phpini httpdconf htaccess