Useful Linux command-line tools to work with PHP projects

Linux provides a lot of interesting command-line tools that we can use when working with PHP projects. In this post we give you some useful commands:

Find all PHP files in the current directory

$ find . -type f -name "*.php"
./index.php
./vendor/autoload.php

Obviously, we can use the same command to get another types of files, such as TWIG or JSON files:

$ find . -type f -name "*.twig"
./test.twig

$ find . -type f -name "*.json"
./composer.json

Check the syntax of all PHP files in the current directory

By itself, “php -l” checks the syntax of a single PHP file. This command can be really useful as part of a CI (Continuous Integration) system to check that we don’t have any syntax error in all PHP files of our project:

$ find . -type f -name "*.php" -exec php -l {} \;
No syntax errors detected in ./index.php
No syntax errors detected in ./vendor/autoload.php

There are another linters and validators that can be used the same way, but they require additional tools. For example, Composer has an option to validate the “composer.json” file, and Symfony provides linters for Twig and YAML files.

For example, if you have several “composer.json” files in your project and want to validate all of them, you can use the “composer validate” command (or “php composer.phar validate”) to check them all:

$ find . -type f -name "composer.json" -exec composer validate {} \;
./composer.json is valid for simple usage with composer but has
./vendor/symfony/config/Symfony/Component/Config/composer.json is valid
./vendor/symfony/console/Symfony/Component/Console/composer.json is valid
./vendor/symfony/dependency-injection/Symfony/Component/DependencyInjection/composer.json is valid

Get the size of each Composer dependency

How big are your dependencies? The du command estimates the file space usage for each dependency:

$ du -h -d 2 vendor
 56K	vendor/composer
496K	vendor/symfony/config
1.2M	vendor/symfony/console
...

Find suspicious PHP files

It’s very common in hacked WordPress projects to inject malicious code that makes use of functions such as eval(), exec() or base64_decode(). The following command finds PHP files that contain any of these functions:

$ find . -type f -name "*.php" -exec grep --with-filename "eval(\|exec(\|base64_decode(" {} \;
./virus.php: eval(base64_decode($_GET['a']));

Get the number of “use” statements in all PHP files, ordered

Does your classes contain too many “use” statements? Maybe it’s time to refactor! Check candidates with the following command:

$ find . -type f -name "*.php" -exec grep --with-filename -c "^use " {} \; | sort -t ":" -k 2 -n -r
./vendor/symfony/console/Symfony/Component/Console/Application.php:28
./vendor/symfony/dependency-injection/Symfony/Component/DependencyInjection/Dumper/PhpDumper.php:21
./vendor/symfony/console/Symfony/Component/Console/Tests/ApplicationTest.php:18
./vendor/symfony/finder/Symfony/Component/Finder/Finder.php:15
...

Find files with abstract classes

The following command will find all files that define abstract classes:

$ find . -type f -name "*.php" -exec grep --with-filename -c "^abstract class " {} \; | grep ":[^0]"
./vendor/twig/twig/test/Twig/Tests/Profiler/Dumper/AbstractTest.php:1
./vendor/twig/twig/lib/Twig/TokenParser.php:1
...

Similarly, we can find PHP files containing 2 or more classes in a single file, ordered:

$ find . -type f -name "*.php" -exec grep --with-filename -c "^class " {} \; | grep ":[2-99]" | sort -t ":" -k 2 -n -r
./vendor/twig/twig/test/Twig/Tests/Node/ModuleTest.php:4
./vendor/twig/twig/test/Twig/Tests/IntegrationTest.php:4
...

List PHP settings for the xdebug extension

“php -i” outputs a huge list with all the settings that are in use. Thanks to the grep utility, we can display only the ones we are interested in. For example, to get all settings related to the xdebug extension:

$ php -i | grep xdebug

Find empty files and/or directories

Check if you left empty files in your project:

$ find . -type f -empty
./test.txt

List files currently open by a PHP process

In Linux, all processes are mapped into the /proc filesystem, so we can get lots of interesting information from a process, like open files.

For example, in the following script we open a file called “test.txt” located in the same directory of the file and then wait for 100 seconds so we have time to execute some commands.

$file = fopen(__DIR__ . 'test.txt', 'w');
sleep(100);
fclose($f);

Then, we execute it and get the PID (process id):

$ php test.php &
[2] 9525

$ ps aux|grep "test.php"
root      9525  0.1  2.0 161404 10736 pts/0    S    11:07   0:00 php test.php

So, the PID is 9525. Now, we can look for information about the process in the /proc filesystem. Each process has a directory with the same name as the PID. To get the list of open files:

$ ls -l /proc/9525/fd
total 0
lrwx------ 1 root root 64 Apr 23 11:07 0 -> /dev/pts/0
lrwx------ 1 root root 64 Apr 23 11:07 1 -> /dev/pts/0
lrwx------ 1 root root 64 Apr 23 11:07 2 -> /dev/pts/0
lr-x------ 1 root root 64 Apr 23 11:07 3 -> /proc/9525/auxv
l-wx------ 1 root root 64 Apr 23 11:07 4 -> /test.txt

The “fd” directory means file descriptors. And we can see that “test.txt” is there! The other file descriptors are for standard input, output and error.

We can also get the the full path of the command, so we know that has been executed using PHP 5.4:

$ ls -l /proc/9580/exe
lrwxrwxrwx 1 root root 0 Apr 23 11:11 /proc/9525/exe -> /usr/local/php54/bin/php

$ /usr/local/php54/bin/php -v
PHP 5.4.5 (cli) (built: Jul 26 2012 15:17:07)

Based on these commands, we can create new ones to fit our needs.

abril 23 / 2015

Upcoming Conferences

WeCamp

PHP New Zealand

PHP Summer Camp Croatia

PHPNE

MadisonPHP

brnoPHP

SymfonyLiveLondon

ZgPHP

PHPSouthAfrica

PHPNWUK

SymfonyLiveNYC

PHPForumParis

PHPARG

PHPWorld

TechMeetupUY

SymfonyConMadrid

Symfony2 components overview: Filesystem

The 15th post of the Symfony2 components series is focused on the Filesystem component, which provides some basic utilities to work with the filesystem. It extends PHP built-in functions such as mkdir() or copy() to make them more portable and easier to use and test.

The Filesystem component provides basic utilities to deal with the filesystem

The Filesystem component provides basic utilities to deal with the filesystem

The problem

PHP built-in functions for filesystem operations mimic common utilities in Unix-like based systems to work with the filesystem. They borrow not only their names, but also the way they work. Because of this, their usage is not very intuitive, especially for Windows users. Take this code as an example:

mkdir(__DIR__ . '/dir/subdir');

First, you may not have permissions to create the directory “subdir” inside “dir”:

PHP Warning:  mkdir(): Permission denied in filesystem_test.php on line 5

Also, it may happen that you do have permissions, but the directory “dir” does not exist:

PHP Warning:  mkdir(): No such file or directory in filesystem_test.php on line 5

Both errors are pretty annoying, as they throw PHP warnings that cannot be intercepted. Also, there is no good reason (at least for me) to force us to create first “dir” and then “subdir”. By using the Filesystem component we can get rid off from all these annoying issues. They work like you would expect without reading the docs and take care of the small (or not so small) differences between Windows and Unix-like systems.

use Symfony\Component\Filesystem\Filesystem;
use Symfony\Component\Filesystem\Exception\IOExceptionInterface;

$filesystem = new Filesystem();

try {
    $filesystem->mkdir(__DIR__ . '/dir/subdir');
} catch (IOExceptionInterface $e) {
    echo 'Error: ' . $e->getPath() . ' could not be created';
}

Yay! The mkdir() method of the component creates both directories recursively.

Another advantage is that we can get exceptions when something goes wrong, so if we try to create a directory in a place that we don’t have permissions, we will get an IOExceptionInterface:

$filesystem->mkdir('/root/dir/subdir');
Error: /root/dir/subdir could not be created

In general, methods accept strings, arrays or objects implementing the Traversable interface, so we can create several directories (or any other operation) in a single call. Remember that the Traversable interface is an abstract interface, IteratorAggregate or Iterator extend from it.

The following snippets are equivalent:

$filesystem->mkdir(__DIR__ . '/dir/subdir1');
$filesystem->mkdir(__DIR__ . '/dir/subdir2');
]);
$filesystem->mkdir([
    __DIR__ . '/dir/subdir1',
    __DIR__ . '/dir/subdir2'
]);
$dirCollection = new DirCollection([
    __DIR__ . '/dir/subdir1',
    __DIR__ . '/dir/subdir2'
]);
$filesystem->mkdir($dirCollection);

The DirCollection object is just a toy class implementing the Iterator interface:

class DirCollection implements Iterator
{

    protected $directories;

    protected $position;

    public function __construct(array $directories = [])
    {
        $this->directories = $directories;
        $this->position = 0;
    }

    public function current()
    {
        return $this->directories[$this->position];
    }

    public function next()
    {
        ++$this->position;
    }

    public function key()
    {
        return $this->position;
    }

    public function valid()
    {
        return isset($this->directories[$this->position]);
    }

    public function rewind()
    {
        $this->position = 0;
    }

}

We could also have used built-in iterators. For example, we could change the modification time of all files and directories belonging to a given directory recursively combining RecursiveDirectoryIterator and RecursiveIteratorIterator. With RecursiveDirectoryIterator we iterate recursively over filesystem directories and with RecursiveIteratorIterator we go over the whole hierarchy.

$filesystem = new Filesystem();

$directoryIterator = new \RecursiveDirectoryIterator(__DIR__ . '/origin', \FilesystemIterator::SKIP_DOTS);
$iterator = new \RecursiveIteratorIterator($directoryIterator, \RecursiveIteratorIterator::SELF_FIRST);

$filesystem->touch($iterator);

Useful methods

We have seen so far only two methods: mkdir() and touch(), but the component provides a few more. As they are well documented in the official docs, we’ll just list them here and try only a few of them:

  • copy($originFile, $targetFile, $override = false)
  • mkdir($dirs, $mode = 0777)
  • exists($files)
  • touch($files, $time = null, $atime = null)
  • remove($files)
  • chmod($files, $mode, $umask = 0000, $recursive = false)
  • chown($files, $user, $recursive = false)
  • chgrp($files, $group, $recursive = false)
  • rename($origin, $target, $overwrite = false)
  • symlink($originDir, $targetDir, $copyOnWindows = false)
  • makePathRelative($endPath, $startPath)
  • mirror($originDir, $targetDir, \Traversable $iterator = null, $options = array())
  • isAbsolutePath($file)
  • dumpFile($filename, $content, $mode = 0666)

The dumpFile() method is especially interesting, as dumps atomically content into a file. That means that we will never see a partially-written file, as it writes a temporary file first and then moves it to the new file location when it’s finished.

Finally, the makePathRelative() method is really useful when we want to have a relative path based on our current working directory.

$dir = '/home/raulfraile/filesystem/';

echo $dir; // /home/raulfraile/filesystem/
echo $filesystem->makePathRelative($dir, '/home/raulfraile'); // filesystem/

File locking

From 2.6, the component ships with another interesting feature: a simple file locking mechanism. You may want to implement a locking system for several reasons, but it’s common for avoiding overlapping in cron jobs.

For example, if we have a task that is executed every 5 minutes and those tasks take more than 5 minutes to finish, they will overlap. In a locking mechanism based on files, the first process creates the lock file, so other processes, when check that that file exists, will wait or finish. Once the first process finishes, it removes the lock file.

Its usage is very simple. The following code shows how to create a lock called “cron.lock”, so only one process is running at a time.

use Symfony\Component\Filesystem\LockHandler;

$lockHandler = new LockHandler('cron.lock');
if (!$lockHandler->lock()) {
    // another process is running, exit
    echo 'Another process is already running';

    return 1;
}

// do stuff...
sleep(100);

If we try to execute two processes, we get the error message in the second one:

$ php index.php &
[1] 10241

$ php index.php
Another process is already running

Who’s using it?

More info

Photo: Filesystem, by kleuske.

abril 22 / 2015

Upcoming Conferences

WeCamp

PHP New Zealand

PHP Summer Camp Croatia

PHPNE

MadisonPHP

brnoPHP

SymfonyLiveLondon

ZgPHP

PHPSouthAfrica

PHPNWUK

SymfonyLiveNYC

PHPForumParis

PHPARG

PHPWorld

TechMeetupUY

SymfonyConMadrid

Melody: One-file PHP scripts with Composer dependencies

Melody is a really handy open-source tool to create one-file PHP scripts with Composer dependencies.

Melody is a tool to execute one-line PHP scripts with dependencies

Melody is a tool to execute one-line PHP scripts with dependencies

Installation

As Melody is just a PHAR file, we only need to download it and include it in one of the directories included in our PATH environment variable, for example at /usr/local/bin/:

$ sudo sh -c "curl http://get.sensiolabs.org/melody.phar -o /usr/local/bin/melody && chmod a+x /usr/local/bin/melody"

Then, if we execute “melody”, we should see something like this:

$ melody
Melody version 1.0 by SensioLabs

Usage:
 [options] command [arguments]
Options:
 --help (-h)           Display this help message.
 --quiet (-q)          Do not output any message.
 --verbose (-v|vv|vvv) Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug.
 --version (-V)        Display this application version.
 --ansi                Force ANSI output.
 --no-ansi             Disable ANSI output.
 --no-interaction (-n) Do not ask any interactive question.

Available commands:
 help          Displays help for a command
 list          Lists commands
 run           execute a script
 self-update   Update melody.phar to the latest version.

Simple example

Let’s create a simple example to understand how Melody works. The following script makes use of the symfony/finder component to look for PNG files in the current directory (and subdirectories):

<?php
<<<CONFIG
packages:
    - "symfony/finder: ~2.6"
CONFIG;

$finder = Symfony\Component\Finder\Finder::create()
    ->in(__DIR__)
    ->name('*.png')
;

foreach ($finder as $file) {
    echo $file, "\n";
}

As you can see, it’s a regular PHP script with a heredoc string at the beginning, which defines the packages the script depends on, using the YAML format in a similar way as we do it in composer.json files.

Melody reads the heredoc string called CONFIG and downloads the packages behind the scenes using Composer. To run the script:

$ melody run example.php
/Users/raulfraile/logo.png
/Users/raulfraile/photo.png

Extra arguments

It is possible to send additional parameters to the PHP script appending them after the name of the script so they can be read using the $argc and $argv variables. For example, to configure the directory to look for PNG images:

$finder = Symfony\Component\Finder\Finder::create()
    ->in($argv[1])
    ->name('*.png')
;

And then:

$ melody run example.php /tmp
/tmp/image.png

PHP options

What if you need to pass additional options to the PHP binary? For example, you may want to start the built-in server or change some settings. This can be done using the “php-options” setting.

In the following example, the memory_limit setting has been set to 256M and the built-in server will be launched:

<<<CONFIG
packages:
    - ...
php-options:
    - "-d"
    - "memory_limit=256M"
    - "-S"
    - "localhost:8000"
CONFIG;

Running gists

Melody has another nice feature, it can run GitHub gists directly! This can be useful for scripts that we want to execute in different computers. For example, I created a simple script to check your ServerGrove’s mail settings sending a test email based on servergrove/email-test. To execute it, you need to include 3 parameters: $from, $to and $password:

$ melody run https://gist.github.com/raulfraile/8b87792f1ecbca520191 from@test.com to@test.com 123456
OK

This script sends a test email from from@test.com to to@test.com using the ServerGrove’s mail server. Obviously, it will only work if you are using our mail server, but setting up a different one is trivial.

Conclusion

Melody can be really handy. At ServerGrove, we are thinking about creating a repository of small tools to help our clients to check and manage their servers. What do you think?

Photo: “Music box 2″, by grfx_guru.

abril 15 / 2015
Author Raul Fraile
Category PHP, Tutorials
Comments No Comments

Upcoming Conferences

WeCamp

PHP New Zealand

PHP Summer Camp Croatia

PHPNE

MadisonPHP

brnoPHP

SymfonyLiveLondon

ZgPHP

PHPSouthAfrica

PHPNWUK

SymfonyLiveNYC

PHPForumParis

PHPARG

PHPWorld

TechMeetupUY

SymfonyConMadrid

Symfony2 components overview: OptionsResolver

In the 13th post of the Symfony2 components series we will be talking about one little but extremely useful component: OptionsResolver. This component helps us to reduce the boilerplate code required to create an options system with default parameters. As stated in the official docs, is array_replace on steroids.

9468151425_1ca4b21cb0_z

Different ways to set options? The OptionsResolver component can help you

The problem

I am pretty sure this code sounds familiar to you:

$options = [
    'page' => isset($input['page']) ? $input['page'] : 1,
    'items' => isset($input['items']) ? $input['items'] : 10
];

Here, we want to have an array with options that come from the user (for example, from a RESTful API). And if any of the required options is not present, we use default options. For this, we need to check if it’s present and use the default value otherwise. All this boilerplate is difficult to read and very repetitive.

An alternative would be to use the array_replace function:

$options = array_replace([
    'page'  => 1,
    'items' => 10
], $input);

The array_replace function replaces elements from the second argument into the first one, so if $input contains a different value for any of the options, it would be overwritten.

While this solution is good enough, the OptionsResolver component provides a few interesting features. In addition to dealing with default values, it can validate and normalize the data.

Simple example

The previous example using OptionsResolver would look like this:

use Symfony\Component\OptionsResolver\OptionsResolver;

$resolver = new OptionsResolver();

$resolver->setDefaults([
    'page' => 1,
    'items' => 10
]);

$options = $resolver->resolve($input);

Once executed, $options will contain a plain array with two elements: ‘page’ and ‘items’. If $input contains values for any of these values, they will be overwritten. Let’s see a few examples:

$input = []; // 'page' => 1, 'items' => 10
$input = ['page' => 2]; // 'page' => 2, 'items' => 10
$input = ['page' => 2, 'items' => 20]; // 'page' => 2, 'items' => 20
$input = ['other' => 5]; // UndefinedOptionsException

That’s right! If we try to use a value other than the ones defined, it throws an exception. This would be the first advantage of using the component over array_replace, but there are more.

It is also possible to use a closure to set default parameters in case they depend on something else. We can even set default parameters based on other parameters. For example, let’s add a third parameter, ‘order’, and define the order randomly. You know… improving UX :)

$resolver->setDefault('order', function (Options $options) {
    $orders = ['asc', 'desc'];

    return $orders[rand(0, 1)];
});

As you can see, the function receives an Options object, so we can use it to generate default values based on other parameters. Imagine that we have also a ‘order_by’ parameter, so we could set the order based on this:

$resolver->setDefault('order', function (Options $options) {
    if ('creation_date' === $options['order_by']) {
        return 'desc';
    }

    return 'asc';
});

Validation

We may want to validate input data, something more than necessary when dealing with user-provided data. The component provides some ways to validate data by type or by value using the methods setAllowedTypes() and setAllowedValues().

With setAllowedTypes(), it is possible to restrict the value of the option to the given types. For example, to restrict ‘page’ values to integer values:

$resolver->setAllowedTypes('page', 'int');

It is also possible to allow more than one type:

$resolver->setAllowedTypes('page', ['int', 'float']);

What if we want to constrain the range of values that can be used? Using the method setAllowedValues() we can define a set of allowed values. For example, to limit the possible values of ‘items’ to 10, 20 or 40:

$resolver->setAllowedValues('items', [10, 20, 40]);

But setAllowedValues() is much more powerful, as it can receive a Closure. In the following example, we can validate that ‘page’ is between 1 and 10:

$resolver->setAllowedValues('page', function($value) {
    return $value >= 1 && $value <=10;
});

If you read the post about the Validator component, you know that there is already a constraint to define ranges. Integrating the validator component is really easy:

use Symfony\Component\Validator\Validation;
use Symfony\Component\Validator\Constraints\Range;

$resolver->setAllowedValues('items', function($value) {
    $validator = Validation::createValidator();
    $constraint = new Range([
        'min' => 1,
        'max' => 10
    ]);

    $violations = $validator->validate($value, $constraint);

    return 0 === $violations->count();
});

Normalization

Finally, we can also normalize data before using the options with the setNormalizer() method. It accepts a Closure that receives the value and the rest of options. For example, we could normalize the ‘page’ option in case that the value provided by the user is over the maximum number of pages:

$resolver->setNormalizer('page', function ($options, $value) use ($maxPage) {
    if ($value > $maxPage) {
        $value = $maxPage;
    }

    return $value;
});

Who is using it

More info

Photo: You Choose Your Path, by James Wheeler

abril 13 / 2015

Upcoming Conferences

WeCamp

PHP New Zealand

PHP Summer Camp Croatia

PHPNE

MadisonPHP

brnoPHP

SymfonyLiveLondon

ZgPHP

PHPSouthAfrica

PHPNWUK

SymfonyLiveNYC

PHPForumParis

PHPARG

PHPWorld

TechMeetupUY

SymfonyConMadrid

New Symfony installer: the fastest way to start your Symfony project

Yesterday, the Symfony team introduced the new Symfony installer. Its main goal is to help developers to create Symfony projects faster.

Until now, installing Symfony to start a new project required a few steps:

  1. Download the code (zip file, git clone, etc.)
  2. Install vendors with “composer install”
  3. Tweak configuration settings

The installer tries to do this in one step. It downloads a compressed file with all the code, including the vendors directory, so you don’t need anything else to run Symfony for the first time.

Installing the installer

The Symfony installer is a PHAR file that runs on PHP 5.4+ (if you are still using PHP 5.3, upgrade!). To install it in your system:

Unix-based systems (Linux, Mac OS)

$ sudo curl -LsS http://symfony.com/installer -o /usr/local/bin/symfony
$ sudo chmod a+x /usr/local/bin/symfony

Windows

c:\> php -r "readfile('http://symfony.com/installer');" > symfony

Create a new project

To create a new project, just execute “symfony new [directory]“:

New project

This will create a new Symfony project using the latest stable version. To use a different version, the command accepts another parameter to use a different version. This parameter can be a branch (it will install the latest version for the branch), a specific version or even “lts” to use the most recent LTS (long term support) version.

Generating a demo project

The tool includes a command to generate a demo project following the official best practices, so you can learn from it.

To generate the project, just run “symfony demo”:

Generate demo project

Then, execute “php app/console server:run” to test the demo project using the PHP built-in webserver. The demo project should look like this:

Demo project

And this is the demo backend:

Demo backend

What about Composer?

Composer is not being replaced. At all. In fact, Composer is being used in the server to download all dependencies before generating the compressed file that you will use. Also, you will still need Composer to add new dependencies or upgrading Symfony.

Future work

They have been working hard during the last few months to create the installer. It works great but it needs some extra features that I am sure will be added in the future, as the first goal was to have a tool easy to use and working in multiple platforms.

  • HTTPS support: currently, the installer downloads compressed files using HTTP, instead of HTTPS. Using HTTPS is highly recommended to avoid man-in-the-middle attacks, as someone might be able to change the contents of the compressed file to inject malicious code.
  • Caching: if you install the same version of Symfony twice, it will be downloaded twice. Adding a cache layer would be interesting, as it would also allow to have some sort of “offline mode”. Have you tried to start a new Symfony project while you are in a train? :)
  • Reduce file size: depending on the developer machine, the installer chooses between two formats: zip and gzip (tar.gz). Much better compression ratios can be achieved with bzip2 and xz. It would make the installer even faster!

Under the hood

The installer uses the Symfony console component to create a command-line application. Then, to create a new project, it chooses the format of the compressed file that will be downloaded using Distill. As the GZIP file is smaller than the ZIP one, it checks if your machine can decompress GZIP files. Files are downloaded with Guzzle and decompressed using again Distill. Finally, it removes some files that are not useful for your project, like LICENSE, UPGRADE*.md and CHANGELOG*.md files, generates a proper random secret value using openssl_random_pseudo_bytes() if available and creates an appropiate .gitignore file.

Photo: “Speed…”, by Rami.

marzo 27 / 2015
Author Raul Fraile
Category PHP, Symfony, Tutorials
Comments 3 Comments

Upcoming Conferences

WeCamp

PHP New Zealand

PHP Summer Camp Croatia

PHPNE

MadisonPHP

brnoPHP

SymfonyLiveLondon

ZgPHP

PHPSouthAfrica

PHPNWUK

SymfonyLiveNYC

PHPForumParis

PHPARG

PHPWorld

TechMeetupUY

SymfonyConMadrid