RSS
 

UTF8 can be tricky – especially with PHP

15 Aug

Everybody uses (or should!) UTF8 these days. An easy and fully supporting PHP version I did not come across yet, though. It seems there is sometimes more to it. This article is supposed to guideline the basic setup of a CakePHP app using UTF8 and will go beyond that to the really tricky parts regarding the de facto standard encoding these days.

Note: this post is really long overdue and was in my draft folder for 2+ years. So here it is, quickly published before it got even more dusty^^ And dusty sure is the right word with (hopefully) no one using ANSI/ISO-8859-1 anymore these days.

UTF8 and PHP

Use the mb_ functions if you know that you real with strings than can contain UTF8 chars. So if you want to count the length of such a string:

$length = mb_strlen($string);

If you are simply manipulating strings, you do not always have to use those slower and UTF8-aware fnctions, though. But in doubt always do so.

UTF8 and preg_match()

Now this is a tricky one – especially if you don’t want to recompile PHP with the PCRE UTF-8 flag enabled or if you don’t know about it at all. IMO that should be the default, but it usually isn’t it seems.

Most times, when dealing with UTF8 strings, the /u modifier and p{L} helps:

preg_match('/^\p{L}[\p{L} _.-]+$/u', $username, $matches)

In other cases you might have to add (*UTF8) in your pattern.

UTF8 and CakePHP

CakePHP setup

The main parts are handled in the book, especially in the getting-started section. But the main part that sometimes people get wrong is that the APP encoding is “utf-8″ while in the database.php its spelled utf8.

Make sure you save all files as “UTF8 without BOM” via your IDE as soon as they start to contain UTF8 chars. Failing to do so will cause output issues. I usually try to avoid this and use Locale translation and mainly English chars in all files as much as possible.

Note: Before adding any UTF8 chars to files, those files are always ANSI (there is no way without the BOM to distinguish those two encoding formats as they are one and the same here). So no matter how often you try to save them as UTF8, they will always still be ANSI. In case you wondered why it falls back to it again in most IDEs.

Correcting PHP functions

Some PHP functionality has been wrapped in CakePHP to overcome deficiencies regarding Unicode. String::wordWrap() for example replaces the faulty wordwrap() function.

I also added a few fixes to my Tools plugin as Utility/Utility class: – pregMatch(): Unicode aware replacement for preg_match() – pregMatchAll(): Unicode aware replacement for preg_match_all() – strSplit(): Unicode aware replacement for str_split() – pregMatchAll(): Unicode aware replacement for preg_match_all()

Probably more to come..

Proper validation

Make sure your validation is unicode aware – that’s probably one of the most made mistakes from mainly English speaking devs/people. They maybe assume that it works to simply use strlen() or a [a-z] regex or alike – not taking into account that for example many normal first/last names contain a lot of special chars. Validation here should never be too strict. Otherwise a lot of people will be very upset.

So in the above example we do NOT want to use

preg_match('/^\[a-z][a-z .-]+$/i', $firstName)

but something more like

preg_match('/^\p{L}[\p{L} .-]+$/u', $firstName)

to validate a first name. IF we actually have to validate this further than a simple “not empty” check is a different topic (I don’t think so). But if you really must, PLEASE do not shut people out because their parents gave them non-English names ;)

A similar thing I had to fix in the core a while back, regarding domains/urls. And this is CakePHP2.5 – the current master – so that topic sure is still quite current for some cases. More and more so with further internationalization.

Checklist for your CakePHP app

  • Ideally, use utf8_unicode_ci as collasion for your DB
  • Your layout should contain <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  • The apache/nginx should serve files as UTF8 via header Content-Type text/html; charset=UTF-8

Outview

Only in PHP7 (as PHP6 got skipped) there will be a more built-in approach then for UTF8. Until then (and maybe even then) we will have to fight quite a lot here for the next years.

There are even a few popular projects in GitHub around the UTF8 issues, e.g: – https://github.com/nicolas-grekas/Patchwork-UTF8 – https://github.com/heartsentwined/php-utf8 – https://github.com/voku/portable-utf8 – https://github.com/gatsu/UTF8

Might be worth checking out.

Anything missing? Please let me know.

2 votes, 5.00 avg. rating (97% score)
 
3 Comments

Posted by Mark in CakePHP

 

All new CakePHP Tips Summer 2014

05 Aug

For CakePHP2.x

Awesome CakePHP

Check out this all new awesome CakePHP list: awesome-cakephp Star it, fork it, enhance it :)

Note the cake3 branch that will proabably soon be filled rapidly with all new shiny CakePHP3 resources and plugins.

There are also a few more generic awesome lists links in there.

CakeFest 2014 coming up

Check it out – and be part of it. You can read about last year’s event here.

This year is going to be awesome (again that word^^). And not only regarding the temperatures in Madrid ;) A few well connected people like @philsturgeon pulled some strings and the event is listed on sites like php.net. Let’s make this the largest fest ever.

Note: My attendance app only lists a very small subset of all people actually joining the event!

Finally a book on CakePHP2.5+

CakePHP Core Developer @josegonzalez wrote a cool book on CakePHP: Rapid Application Development with CakePHP 2. It has some real insight in topics that are not covered by the online CakePHP documentation.

Hot tip: Use the promo code DEREUROMARK when purchasing the bundle to get 20% off your purchase!

Deployment with composer and script hooks

This is already widely used in Symfony2, for example. Here it fires the following scripts after installing/updating to clear the cache and re-build assets etc:

"scripts" : {
    "post-install-cmd" : [
        "Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
        "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
        "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
        "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
        "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
    ],

The list of available commands be found here: getcomposer.org/doc/articles/scripts.md

Why not leveraging this for your Cake2.x app, as well? You could re-built AssetCompress files, clear the persistent/model cache, run custom post-update hooks and so on. (DB) Migration could also be hooked in this way.

Post links inside Forms

Most probably didn’t see this change. But it allows postLink() to be used inside forms now. The issue was, that forms don’t work inside forms. And that post links contain those inline, breaking the form around it.

You can now easily create “Delete” links or other POST links inside your forms. It can write the forms to a View block which you can output later on then. All you need to do is set 'inline' => false:

echo $this->Form->postLink('Delete', array('action' => 'delete', 1), array('inline' => false));

And output your buffered View block after closing the form:

echo $this->View->fetch('postLink');

This also works with multiple postLink() calls. The View block would simply contain all of the forms then.

Prevent internal field names

I try to stay away from internal framework used names for fields. Once you attach Search plugin or any other additional functionality on top of your actions, this could get messy otherwise. The Paginator, for instance, uses “limit”, “sort”, “direction”, “page”. In order for them to not conflict with search fields on existing fields, I avoid them and use “order” instead of “sort” or “max” instead of “limit” for field names. In case this is not avoidable or a legacy DB schema, one can add the model prefix, though: “MyModelName.sort” and would also be able to resolve the conflict. But if you can already do so in hindsight it is probably smarter.

Array and ArrayObject

The new CakePHP3 core uses ArrayObject already quite a bit. In 2.x you probably don’t use it too much yet. But it is wise to voice a warning regarding on some flaws of ArrayObject. This article describes it. Basically, empty() and ArrayObject don’t play nice. It only works with count():

if (count($options)) {
    // Yes
} else {
    // No / empty
}

The alternative would be to cast it prior to checking on it: (array)$options. Then empty() would work again.

The rest of those issues in the article are also quite interesting. Check them out, as well.

A complete list of “flaws” has been put together on this page. It reads like the bible of Anti-PHP, tough :) Quite entertaining even in some parts. I would just read it as an informative page, though. It is good to know those things, but it is also easy to avoid or work-around them.

MySQL and returning integers

Some probably wondered why integers and other non-stringish values is often returned as string. This has nothing to do with CakePHP itself, but with the PDO extension of PHP underneath.

But you can easily disable that with the following flags in your database.php config:

'flags' => array(
        PDO::ATTR_STRINGIFY_FETCHES => false,
        PDO::ATTR_EMULATE_PREPARES => false
    )

PHPUnit and Windows

You might have seen something as descibed in this issue: Weird output in your console when running unit tests. This is mainly because the default setting for PHPUnit in the phpunit.xml(.dist) file is usually colors="true". You can easily overwrite it in your own file and the weird output stops:

<phpunit
  colors="false"
  ...

Or you can try to use the library linked in that ticket. I didn’t try that one yet.

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in CakePHP

 

Migrating SVN apps to Git and Composer

23 Jul

Just recently I had to move a few remaining legacy CakePHP2.x apps from SVN to Git+Composer. This serves as a short how-to.

You can also use this post if you are migrating a git app to a composer based git app.

Note: Installing a new CakePHP 2.x app with Git and Composer right away is dead easy and explained quite well in the cookbook. This post is more about migrating existing ones.

SVN + Composer/Git?

They can actually work side-by-side. For a migration process this can be useful, especially when migrating huge apps. Then doing it piece by piece helps to avoid chaos.

For this little how-to the main repo will stay a SVN one until the very end for this very same reason.

First steps

We have our SVN repo – up to date thanks to svn update. Download composer.phar in your APP dir. This file should be added to your excludes. Set up a composer.json file in there, as well, and commit it.

CakePHP Core

You might have hard-coded the core files in ROOT/lib/Cake – or used an svn:externals on ROOT/lib. Either way, remove that and add CakePHP2.x as Composer dependency.

Update index.php and webroot.php

As per documentation, we now need this in those files:

define('CAKE_CORE_INCLUDE_PATH', ROOT . DS . APP_DIR . DS . 'Vendor' . DS . 'cakephp' . DS . 'cakephp' . DS . 'lib');

Resolve global vendors folder

In case you used a global vendors folder it will now not automatically be included anymore. To quick-fix this, add this to your bootstrap.php (at the beginning probably):

App::build(array('Vendor' => array(ROOT . DS . 'vendors' . DS)));

This will load your App::import()ed classes again from that folder. Same goes for the plugins folder, by the way.

In a second step you can then start moving those to the APP/Vendor folder – either as Composer dependency or hard-coded.

When all are transferred, you can remove the global vendors folder and also remove the additional App::build() call.

Resolve svn:externals for plugins

Most plugins will probably be hard-coded or included via svn:externals. Remove the svn:externals and add them as Composer dependency. Do the same for any plugins in your ROOT/plugins folder. If they are hard-coded there, move them to the APP/Plugin folder.

Almost done

At this point we already have a fully functional application again that loads all dependencies via composer – even though it is still a SVN repo. All it needs is

cd .../root && svn update
cd app && composer install

If some global vendor libs are harder to migrate or the “svn to git” conversation is troublesome, you can take your time get it up and running again on this level just fine. At some point you will have finished the migration though, then proceed to the final step.

Move SVN to Git

When all externals have been resolved, the root vendor and plugin folders have been cleaned and their content moved to the APP pendents, we can now start migrating the application to Git. For this we drop the usual /trunk/... folder structure and directly copy the APP folder from your SVN repo into the root of your new Git repo. The ROOT folder itself serves no purpose anymore, and we can have APP as root folder now. Therefore in root, you should now have the composer.json file. For an example, see cakephp-sandbox.

As for “copy”: You can either directly copy-and-paste your repo (quick and dirty), or use git-svn tools to preserve your changelogs/history. The latter is the cleaner approach, of course. See this guide on how to do that. Some services like GitHub might also offer a git-svn push access or link useful tools. Those approaches usually need a folder move of the APP folder content to the root level then on top.

Either way, the more you composer and the less you hard-code your libs, the quicker and easier the svn…git transformation will go.

Now updating app + core + plugins + vendors is just:

cd .../root
git pull
composer install

Since the APP folder is now the root folder, no need to further cd deeper.

Sugar

You may add

require_once dirname(__DIR__) . '/Vendor/autoload.php';

at the top of your bootstrap.php file to add autoloading via PSR for new namespaced repos (on top of the still working App::import() and App::uses() methods).

That means, that you can now use namespaces and third party packages via composer quite easily. Let’s say we want to use the MediaEmbed lib in our project. We then simply add it to our composer.json:

"require": {
    "dereuromark/media-embed": "dev-master"
}

In our CakePHP classes, e.g. an EmbedHelper, we can now use it right away:

<?php
use \MediaEmbed\MediaEmbed;
 
App::uses('AppHelper', 'View/Helper');
 
class EmbedHelper extends AppHelper {
    /**
     * @param string $url Absolute URL of the video/audio.
     * @return string|null HTML snippet or null on failure.
     */
    public function media($url) {
        $MediaEmbed = new MediaEmbed();
        if ($MediaObject = $MediaEmbed->parseUrl($url)) {
            return $MediaObject->getEmbedCode();
        }
    }
}

Isn’t that awesome? You now have the full PHP5.3+ (GitHub) lib variety out there at your disposal. You might want to check out this list of cool libs.

Private repos

With the externals, it was quite easy to pull from your own repo URLs. Composer makes this a little bit more difficult. To overcome the issue for GitHub private repos, for example, you can use vcs:

"require": {
    ...
    "your-username/cakephp-some-private-plugin" : "*"
},
"repositories" : [{
        "type" : "vcs",
        "url" : "git@github.com:your-username/cakephp-some-private-plugin"
    },
    ...
],

Note that this will prompt once for your username and password and create a token for re-use:

Your GitHub credentials are required to fetch private repository metadata. The credentials will be swapped for an OAuth token stored in …/Composer/auth.json, your password will not be stored.

Last tips

This tutorial assumes that you are properly using the current master (2.5.3 at this time) version of CakePHP. Always upgrade to the current master – never use anything older.

Make sure you have a current index.php/test.php file as well as a current APP/Console dir including those cake scripts (current as in master/2.5.3 at least). You can just copy them over from the CakePHP source folder (Cake/Console/Templates/...) if you are unsure. Especially for console this is quite important. Otherwise Console/cake might not work out of the box with the new composer approach – even though it would if you had up-to-date files.

Also, don’t forget to .gitignore all included dependencies (both Vendor and Plugin ones). And also the ones from SVN, composer.phar etc.

2 votes, 4.50 avg. rating (90% score)
 
No Comments

Posted by Mark in CakePHP

 

HTML, Markdown or BBCode?

16 Jul

Or something different? Or just plain text maybe?

“User input is hard”

There are pretty prejudiced blog posts in favor or against the usage of BBCode for blog posts, comments etc. Here for instance – continuing on this second post.

Personally, I am in favor of some abstraction for the frontend user, usually BBCode or a similar syntax. But it always depends on the usage in the app. For forums, blog post comments, etc they are sure quite useful and avoids exposing HTML to users.

Let’s dig into some options more deeply.

Plain text

We all probably do or did it: Using .txt files to store some pseudomarkup enriched text including lists and headings. While its simple and easy, the output on a a browser for such textual representations is pretty poor. It starts with missing <p> paragraphs or newlines. All those would have to be manually modified. In CakePHP there is a helper method that does exactly that: TextHelper::autoParagraph(). Adding autoLink() you even get your URLs and emails enriched as HTML. It works perfectly for all varchar or text DB fields that are exposed as textareas in the forms and only needs to accept plain text. This is all nice for very basic use cases, but is quite limited in the presentation.

HTML and WYSIWYG Editors

This widely used in backends as this is easy to implement. Just add a JS based WYSIWYG editor on top of your textareas and you got yourself editing in the way it will be presented upon output. This live preview while typing sure is a nice bonus here. One drawback often is the amount of overhead added via tags, classes and alike. But that depends on the editor being used.

Often times there is a “plain HTML” button to even allow custom HTML modifications in the source code. While this is probably the most flexible approach, it is also the most dangerous. You can easily screw up the HTML – invalid HTML is hard to spot and might break your whole frontend layout.

Tools such as htmlpurifier exist to support those editors in removing unwanted content, fixing broken HTML and cleaning up the source code (mess). This also makes it possible to allow a subset of HTML in non-backend textareas untrusted users can have access to. In CakePHP, for example, you can attach a PurifiableBehavior to your model that cleans the content upon saving.

The most commonly used editors are probably listed on this comparison site.

BBCode

That was the very first abstraction level available. A lot of forum software still uses it quite thoroughly.

Everybody who opposes BBCode completely (like the above posts), does not know much about the user perspective. Users don’t care about semantics and just want their link to be added. They want it simple and straight forward. So [img]url[/img] in BBCode would be more intuitive as the resulting <img src="url" /> in HTML. Now what is easier to understand for a newbie? What is easier to read? Of course we have a little bit more processing with BBCode. But with caching that is minimized to nothing. For admin backends it is usually easier to use HTML. This way they have more tags, attributes to chose from. We can also assume that they don’t want to harm the site and that they know what they’re doing.

The second important point is abstraction. [code] can be <code>... or even <pre>... or a combination of both. Everybody understands [code] whereas [pre] etc is not so understandable. So we use [code] and afterwards transform it into the more complex HTML tags we need for markup. But the user text stays clean and straight-forward. He doesn't need to know about the mapping of [code=php] to <pre id="xyz" class="php"><code>...

jbbcode looks like a solid implementation for this.

There is even a WSYIWYG editor for this now. Even though that one is based on JS - meaning you would have to keep your custom rules redundant, once in PHP and once in JS for this editor. Also, there might be slight edge case differences between preview and actual result.

Markdown

This is becoming more and more popular these days. Not only for developers who use GitHub a lot, but also by bloggers using WordPress plugins for this or other websites that want to avoid the HTML overhead when displaying lightweight markup text. The benefit here is that the text can be written almost as normal text. And even non-developers would easily understand lists such as:

- one
- two
- three

So it combines text with leightweight markup that is easily understandable by everyone - and probably even used intuitively without knowing it. Translation into HTML is straightforward.

Currently most people prefer the GitHub flavored addons as the original markdown implementation hasn't had any progress anymore the last years. A nice demo and comparison shows the difference.

A slower but probably more powerful library is Ciconia. It is intended to be more flexible and extendable.

There are nice WSYIWYG editor implementations for Markdown: sofish.github.io/pen or markitup. But as with BBCode: since that one is based on JS - meaning you would have to keep your custom rules redundant again, once in PHP and once in JS for this editor. Also, there might be slight edge case differences between preview and actual result.

Imagine you can write all your HTML in such a DRY and non-HTML-polluted way and still get nice HTML from it. And you can also use it for textual representation right way (e.g. text emails). Awesome.

Speaking of - there are nice tools that can actually take your already written HTML and revert it back into Markdown. See http://blog.oddbit.com/2012/11/06/convert-html-to-markdown/ and to-markdown.

So if you have already existing records or blog posts in the old format, you might be able to convert them and then use markdown only from there on.

Summary

From what we read so far, the complexity would probably be best described as:

Text < Markdown < BBcode < HTML

And the further left, the better - not only for interoperability. You should use the easiest format that suits your need.

HTML

  • No additional parsing needed (once it is validated and saved)
  • Cannot work without a sanitizing process for non-admins (stripping off any unwanted attributes or unsafe elements)

BBCode

  • Simpler sanitizing
  • Abstraction possible ([ video ] or [spoiler] tag)
  • Does not interfere with HTML Markup (e.g. for code snippet posts in dev blogs)

Markdown

  • A compromise between writing plain text and using minimal additional markup to enhance it
  • Intuitive
  • Less error-prone than BBCode regarding simple tags/markup
  • Maybe more error-prone than BBCode for more complex tags/markup
  • Nice for inline references/images/hyperlinks (links can be grouped at the bottom)

Combining them

Sometimes, the lightweight Markdown might not cover everything. Writing a custom wrapper you can easily combine them enhancing your toolset.

You use Markdown as primary parser and parse the remaining BBCodes afterwords. HTML could be allowed using a custom markdown rule, e.g.:

```html
<myhtml />
```

You could also use BBCode then, of course:

[html]
<myhtml />
[/html]

Adding plain HTML in between the Markdown and BBCode markup would work, as well (I do that^^). But this can easily break or have unseen side effects when trying to escape the source code. It is more difficult to distinguish between an HTML tag <b>I am bold</b> and just plain text containing those chars by accident: I like the brand <FooBar>. In that case all those non-HTML-intended characters need to be properly escaped, which really is annoying. So please don't do that ;)

In the example above the whole page would be escaped using h() (the htmlspecialchars() in CakePHP). This way it is secure by default. And the tags from above would automatically undo h() to display the raw HTML again.

I think combining them in a logical order can in some cases make all the difference and solve all your problems at once. You have the simple and lightweight markup as basis, you are able to apply custom codes via BBCode similar syntaxes and on top you can always use real HTML for more complex scenarios (tables and alike) where necessary.

Side notes

This blog also uses Markdown for all posts and some BBCode for the comments, of course.

I did not mention Textile as even though introduced shortly before Markdown it never really became that popular. Those are similar, though, in their ideas. A full list of further lightweight markup languages can be found at wikipedia.org.

Further addons

I use anchorjs to automatically add anchors on the fly using JS. This is especially useful if the parsed markdown itself produces headings without any attributes. This way they are added without having to dig deeper into the post-processing of the markdown parser or modifying the resulting HTML. Another alternative would be anchorific.

You can use my MediaEmbed lib as addon for BBCode or Markdown (or even plain text) to auto embed Video snippets. See the examples/bbcode.php there for a live example.

Further Links and Resources

This BBCode parser once looked quite promising. But it now seems abandoned.

The MarkupParsers Plugin combines several markup syntaxes into a plugin. There are even MarkdownView classes which would render a complete markdown-flavored layout into HTML. For me a helper or lib wrapper usually suffices as I usually only output parts of the layout as such a markup-flavored text.

See dillinger.io/ or stackedit.io for editing Markdown in real time. And there is a cheat sheet to go along with it.

Outview

I will probably add some real life examples and comparisons soon on my sandbox site. Stay tuned.

0 votes, 0.00 avg. rating (0% score)
 
1 Comment

Posted by Mark in HTML, WebDevelopment

 

CakePHP 2.5 arrived + CakePHP Tips 2014

13 May

You probably read my last tip sections. And I started to move some of them to my sandbox app.

But once in a while it might also be nice to publish a few selected tips, here as well.

Oh, and CakePHP 2.5 is out! Get it – now :) See the last chapter of this post about why you really should upgrade ASAP. And it doesn’t matter if you are on 1.x or 2.x.

URLs in CLI

In CLI and your shells there is no HTTP HOST and therefore usually no absolute URLs. But in order to properly work with links/urls (e.g. sending batch emails), we need that to work. So I use Configure::read(‘Config.fullPageUrl’) here. My configs.php file then contains:

$config['Config'] = array(
    'fullPageUrl' =>  http://www.myapp.de // The primary URL
)

In case you have a different domain for local development like http://myapp.local and you want that to be the fullPageUrl, make sure you overwrite the default in your configs_private.php file:

Configure::write('Config.fullPageUrl', 'http://myapp.local'); // Usually defaults to the (live) primary URL

And in the end of my bootstrap file (after including all config files), I simply do:

if (php_sapi_name() === 'cli') {
    Configure::write('App.fullBaseUrl', Configure::read('Config.fullPageUrl'));
}

To test the current domain/fullBaseUrl, you can use my Setup plugin command cake Setup.TestCli router. It will output both relative and absolute URL examples generated by the Router class with your current settings.

So on the live server then it will output http://myapp.local instead of http://localhost when generating Router::url()s in your shells.

Careful with running shells in CLI

Most are probably not aware, but running shells in CLI needs to have a proper user management around them in most cases. Imagine yourself running your apache as www-data (default) and log in as root or any other user not affiliated with that www-data user/role (bad idea). Once you execute a shell and tmp cache data are (re)built, your www-data user cannot access them anymore, losing the ability to cache and triggering a lot of errors. So make sure you only log in with a user that shares the role of www-data at least, so that both can access each others’ generated files.

A popular example is the ClearCache shell which re-builds your cache dirs in debug 0 (when changing files or db schema makes this necessary).

PS: Of course you could also switch to another cache system than the default File engine. But most probably didn’t do that yet, either.

Merging arrays

Ever wondered what Hash::merge(), array_merge and array_merge_recursive have in common – or don’t have in common? Check out these merge comparison examples.

See what the requirements are – and use the appropriate merge method then.

There is also the + operator, which is quite useful when merging flat arrays and string key based options. This is quite commonly used in the core to merge options and defaults:

$defaults = array(
    'escape' => true
):
$options += $defaults;

In this case the $defaults are merged on top of $options, but only if that key has not been specified yet. This kind of merge is really efficient and fast (4-5x faster than array_merge() itself) – but should really only be used if all keys are definitely strings.

Paginating and merging

A propos merging: When setting up paginate settings in your controllers, try to prevent

public function index() {
    $this->paginate = array(...);
}

This way you kill all defaults you might have set via public $paginate or in your extending controllers (AppController’s beforeFilter() callback for example).

So it is better to use:

$this->paginate = array_merge($this->paginate, array(...));
// or
$this->paginate = array(...) + $this->paginate;

In my 2.x code for example I have this snippet in all my AppControllers to have query strings for paginations:

public function beforeFilter() {
    parent::beforeFilter();
 
    $this->paginate['paramType'] = 'querystring';
}

This will only work with proper merging of defaults and custom settings. I prefer the latter because the settings are string based and here the + operator is the fastest and easiest way of doing things. Once the key is already set in your method, the default will be ignored right away (with array_merge() and nullish values this can be different/unexpected).

And remember to not mix controller and component pagination.

Pagination and sort default order

Adjust your bake templates so that some fields like created/modified are ordered DESC instead of ASC per default in the index actions. For those fields the first click on the header row should display them DESC right away as one would then most likely be interested in the latest changes. Same goes for most date fields like “published” as well as fields like “priority”, “rating”, …

That’s how the baked code (or manually adjusted one if done afterwards) could then look like (index.ctp):

<th><?php echo $this->Paginator->sort('name');?></th>
<th><?php echo $this->Paginator->sort('amount');?></th>
<th><?php echo $this->Paginator->sort('priority', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('status', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('publish_date', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('modified', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('created', null, array('direction' => 'desc'));?></th>

Using modified model data in the form again

Some of you might have had the wish of posted data that was modified in the model due to beforeValidate/beforeSave to appear modified in the view again (so that the reason for validation errors might be more clear etc). So let’s say you have a beforeValidate callback to clean the input of a textarea:

public function beforeValidate($options = array() {
    if (isset($this->data[$this->alias]['comment']) {
        $this->data[$this->alias]['comment'] = $this->_clean($this->data[$this->alias]['comment']);
    }
    return true;
}

So in this case it could easily be that _clean() removes some invalid content and thus the minLength rule is suddenly triggered. Which is weird, since we posted at least twice the length of text. To clarify to the user what is going on, one could adjust the error message – but one could additionally return the modified (ready to save) data instead of the actually posted data.

public function add() {
    if ($this->request->is('post')) {
        $this->{$this->modelClass}->create();
        if ($this->{$this->modelClass}->save($this->request->data)) {
            $this->Session->setFlash(...);
            return $this->redirect(array('action' => 'index'));
        } else {
            // Here we assign the modified model data back to the request
            // object and therefore to the view/form
            $this->request->data = $this->{$this->modelClass}->data;
            $this->Session->setFlash(...);
        }
    } else {
        // Default values for the form
        $this->request->data[$this->modelClass]['status'] = true;
    }
 
}

The input field will now contain the content that was served to beforeValidate(). And combined with a good error message this will probably clear things up.

PS: You can also directly use the model’s name instead of {$this->modelClass}, e.g. ‘Comment’.

MySQL – MyISAM vs InnoDB

InnoDB is a little bit more robust as it allows transactions. Especially with CakePHP and “multiple queries” most of the time (per default) this can be quite helpful in keeping the DB in a valid state. Also read myisam-versus-innodb on pros/cons for each.

One additional problem with InnoDB, though: Per default it creates a file that always increases in size, never decreases again. This can soon be a nightmare with backuping when it becomes >> xx GB of size. See how-to-shrink-purge-ibdata1-file-in-mysql how to avoid that by not using that file, and instead using innodb_file_per_table.

Testing

Testing Controllers

I stumbled upon a few issues with testing controllers – especially plugin controllers. For plugin controllers to be testable unfortunately you always need to use $uses, even if it wasn’t necessary due to conventions. So for your Tools.TinyUrls Controller you would need

public $uses = array('Tools.TinyUrl');

Otherwise it would always try to look for the model/fixture in your app dir, which eventually always fails.

Do not forget to use --stderr when testing in CLI. Otherwise all tests that use the session (and which work fine in webtest runner) will fail:

cake test app AllController --stderr

Test Coverage

If you want to generate a HTML overview of all your locale test coverage:

cake test app AllApp --stderr --log-junit tmp/coverage/unitreport.xml --coverage-html tmp/coverage --coverage-clover tmp/coverage/coverage.xml

The report index.html will be in your /tmp/coverage folder.

More on testing – especially controller testing – can be found on the corresponding Tools plugin Wiki page.

Upgrade (deprecated) code

It is always good practice to upgrade to the current master. Not only for CakePHP, but there especially. It will save you a lot of time in the long run, as migration will be easier and faster in small steps instead of one huge step. It will also make it easier to use the new features and more importantly will also come with a lot of fixes and method corrections that the older versions didn’t get anymore. Those outdated versions usually only get security-bugs fixed. So if you look hours for an error that is already fixed in the current master, it was just a huge waste of time. I have seen that a thousands times – on IRC and elsewhere.

So in case you are not using the current master (2.5), do it now. Internally, upgrading 2.x is a “piece of cake”. Upgrading from 1.x is also not that big of a deal – just needs a little bit more manual adjustments. For most things you can use the core UpgradeShell as well as my Upgrade plugin.

In case you are already upgraded to 2.5, you can and should also remove deprecated functionality in favor of the right one. Those deprecated things will only add to the file of upgrades for the next 2.x release or 3.x. And using the upgrade shell it is usually just one single command to execute. So for 2.5, you should have removed all the “easy” stuff that will clearly be switched with a different way of doing things as it is mentioned in the migration guide, e.g.

  • loggedIn() in favor of Auth::user(‘id’)
  • CakeRequest::onlyAllow() in favor of CakeRequest::allowMethod()
  • Use first argument (string|array) instead of var args for CakeRequest::allowMethod(), AuthComponent::allow(), etc
  • $title_for_layout in favor of $this->fetch(‘title’) and all other deprecated view vars

From 2.4 and below there are also a few left overs that could easily be corrected:

  • FULL_BASE_URL, DEFAULT_LANGUAGE, IMAGES_URL, JS_URL, CSS_URL to Config variables
  • Remove CAKEPHP_SHELL, IMAGES, JS, CSS usage
  • Simplify HtmlHelper::css()
  • Remove Sanitize class usage in favor of a more sane approach
  • Simplify CakeRequest and PaginatorHelper usage with param() if applicable
  • Don’t use request shortcuts anymore – like $this->action in favor of $this->request->action etc
  • Get rid of Model::read() calls – in 3.x this will be get() – I use my Tools plugin MyModel::get() method here already for years
  • Use the new Model::save() syntax for options
  • Completey get rid of named params in favor query strings (Use my Tools plugin CommonComponent and Configure value App.warnAboutNamedParams to detect left-overs)
  • Replace all Set calls with Hash calls, as Set is deprecated. Make sure it is tested thoroughly (as the functionality of Hash might be slightly different).
  • Prevent using Validation::range() in favor of a custom validation rule for inclusive ranges to easier migrate to 3.x – or simply use my Tools plugin MyModel::rangeInclusive() method.
  • Further deprecations in favor of the 2.5+ way to do things

and so on. For some details see Tips-Upgrading-to-CakePHP-2.x.

My Tools plugin also contains a few further tweaks that can help ease migration from 2.x to 3.x See the full list on the corresponding Wiki page.

This will help making sure any future upgrade is less painful. Think about it: When you do that now the remaining TODO list will only be half the size and look a lot less intimidating. When the time comes to upgrade to 3.x it will look quite more doable.

My codez is now officially all 2.5 – and as 3.0 ready as it can get :P

And what always helps a lot is to code clean and structured. A code mess will always be difficult to maintain. So use coding standards and enforce them. Use best practice approaches. Refactor once in a while to prevent a mess from happening. Happy coding :)

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in CakePHP

 

CakePHP flash messages 2.0

21 Apr

This is a topic often discussed. What is the best approach? While I might not have the best, I sure have a fully working one since 2008 and CakePHP1.3. I still use it in all 2.x apps. It is easy and does not require a lot of overhead.

Basically, it enhances the core one with the following features

  • Different types possible (now in 2.x kind of in the core, as well, using elements)
  • Multiple flash messages per type possible (with a max limit to avoid session flushing)
  • Transient messages (via Configure) and real ones (via Session)
  • Transient ones can also be fired from the views (to display some hint/info for this page) – although one should try to stick to the controller

Demo

sandbox/examples/messages

How does it work?

We attach the Common Component and the Common Helper to the AppController:

public $components = array(..., 'Tools.Common');
public $helpers = array(..., 'Tools.Common');

In our layouts we need to switch the default output to ours:

echo $this->Common->flash();

And we can use `em:

public function add() {
    ...
    if ($this->Entry->save()) {
        $this->Common->flashMessage('The Entry has been saved.', 'success');
    } else {
        $this->Common->flashMessage('The Entry could not be saved. Please check the form.', 'error');
    }
}

Styling

We style our flashmessages via CSS

div.flash-messages {
    width: 90%;
}
div.flash-messages div {
    padding: 10px;
    padding-left:40px;
}
div.flash-messages div.error {
    width:100%; border-style: solid; border-width:1px; border-color:#B84D17; 
    margin-right:2px; color:#000000; margin-bottom:8px;
    background: url(/img/css/layout/icon_error.gif) left center no-repeat;
    background-color:#F7C6A5;
}
div.flash-messages div.warning {
    width:100%; border-style: solid; border-width:1px; border-color:#D0C130; 
    margin-right:2px; color:#000000; margin-bottom:8px;
    background: url(/img/css/layout/icon_warning.gif) left center no-repeat;
    background-color:#F6F3A4;
}
div.flash-messages div.success {
    width:100%; border-style: solid; border-width:1px; border-color:#009900; 
    margin-right:2px; color:#000000; margin-bottom:8px;
    background: url(/img/css/layout/icon_success.gif) left center no-repeat;
    background-color:#A5F7A8;
}
div.flash-messages div.info {
    width:100%; border-style: solid; border-width:1px; border-color:#cccccc; 
    margin-right:2px; color:#000000; margin-bottom:8px;
    background: url(/img/css/layout/icon_info.gif) left center no-repeat;
    background-color:#ffffff;
}

Or any other layout for that matter. The images are small icons and also optional, of course :)

Transient flash messages

You can also put flash messages on top that are not stored in session but Configure (for this request only). This can be useful if you don’t intend to redirect and don’t want them to show up if that happens.

From your controller:

$this->Common->transientFlashMessage('This page is currently being redesigned', 'info');

Or from your view ctp (even elements, blocks or the layout if it happens prior to the flash message output):

$this->Common->addFlashMessage('This page is under maintenance. It may be broken!', 'warning');

Output in a specific order or only specific types

You can filter the output, both in order and types:

// Using Common helper
echo $this->Common->flash(array('warning', 'error'));

In this case it would only output the warning and error messages, in this order (Usually the order is “error, warning, success, info”).

Details

For details please the Wiki page.

Outview

It would probably be nice to add element support at some point. This would allow an easier approach to customization of those messages.

There are also quite a few CakePHP core tickets open regarding flash message enhancements – see this or this which might even lead to an own component for it some day. But until then my approach will still be used in all my xx apps :) So in 3.x there will be a Flash component and Flash helper to provide a clean way to produce flash messages. If they provide the same features my implementation currently does, will have to be investigated. But it will use templating which will sure be nice.

0 votes, 0.00 avg. rating (0% score)
 
3 Comments

Posted by Mark in CakePHP

 

Update your Server to PHP5.4+

13 Apr

With many new features and more speed it makes sense to upgrade to PHP5.4. It is also a requirement when you start using CakePHP3.0 (the dev releases) or code that contains short array syntax. So the sooner you upgrade the better.

The following examples are for Debian Squeeze.

Backup

Make sure you have a backup of your server/files/db and your php.ini and other config files.

Updating

Debian doesn’t offer this natively. We need to add some special packages for this.

Open the sources list using nano, vi or the edit tool of your choice:

vi /etc/apt/sources.list

And add those two lines at the end:

deb http://packages.dotdeb.org stable all
deb-src http://packages.dotdeb.org stable all
 
deb http://packages.dotdeb.org squeeze-php54 all
deb-src http://packages.dotdeb.org squeeze-php54 all

We need a public key to avoid those warnings showing up:

wget http://www.dotdeb.org/dotdeb.gpg
cat dotdeb.gpg | apt-key add -

Use apt-get update and apt-get install php5.

That should take care of it. PHP5.4.x should now be installed and ready to work with. And have fun playing around with CakePHP3.0 or some new features of PHP5.4 :)

Keep other packages up to date

Also make sure other packages are up to date using apt-get upgrade.

If it doesn’t install right away, one might also need apt-get dist-upgrade. Careful with this, though. It might install/overwrite more than it should.

Dev Libraries

If php5-dev libraries are needed, they can be installed via apt-get install php5-dev.

Key Benefits of 5.4

  • Speed/Stability
  • Traits
  • Closures and $this
  • callable as typehint
  • Short array syntax (no mixin of method brackets and array brackets anymore)

The only real benefit coming in 5.5 is “Generators” which will allow handling of huge arrays and larger amount of data more gracefully. But other than that 5.4 is just fine for the moment.

Features of 5.3 you can now also use in Cake3

  • Namespaces
  • Closures
  • Late Static Bindings
  • Ternary shorthand form
  • Magic method __callStatic()
  • Nested exceptions
  • These features weren’t used in Cake2 core (and probably most apps) so far due to the BC for PHP5.2. If you already updated your minimum requirement for your server and application, those are also now of great use.

Further tips

The encoding even in more current verions of apache usually defaults to the old “ISO-8859-1″ one. You should make it “utf-8″ per default in your php.ini:

; PHP's default character set is set to empty.
; http://php.net/default-charset
default_charset = "utf-8"

Then even error pages or non-normal responses will be properly displayed. It is sad that even 2014 and in PHP5.4 the default is still not a proper one – probably due to BC or something.

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in PHP

 

Generating PDFs with CakePHP

08 Apr

The recommended approach is to use the CakePHP plugin CakePdf for this. With that plugin it is “a piece of cake” to output HTML views as PDF or render them to files for download/emailing etc.

You can obviously just use an approach just like this article describes. But this makes the code quite dependent on the actual rendering library used isn’t very DRY when there is more than a single page that needs PDF rendering.

Note: This article is about generating PDF files from HTML templates. If you already have PDF files you just want to output or serve as download, then this article is not for you. See serving-views-as-files-in-cake2 then.

Demo

A live demo can be found in my sandbox app. It also helps to compare the different engines.

Installation

First you need to decide on what engine you want to go with.

WkHtmlToPdf is the fastest, but it also depends on CLI and the binaries being installed properly. So it might work differently or not at all on different OS. But using binary files it should work out of the both on both UNIX and Windows. The latter just needs a single exe file drag-and-dropped on your system. wkhtmltopdf.org/downloads.html contains such a ready-to-use binary. You can then link it in your Configure set-up for CakePdf:

Configure::write('CakePdf.binary', YOUR_PATH_TO . 'wkhtmltopdf\wkhtmltopdf.exe');

For UNIX those precombiled binaries on that download page might not work and a simple apt-get install wkhtmltopdf usually won’t do the trick. You need to manually compile it in this case. But on most systems it should be fine. Don’t forget to adjust the path to the binary here, as well, if it is not the default one:

Configure::write('CakePdf.binary', YOUR_PATH_TO . 'wkhtmltopdf/bin/wkhtmltopdf');

I usually go with DomPdf as it is quite reliable, and (as PHP code) even though slower than a native approach will work flawlessly with all OS and setups without additional dependencies. If speed really is an issue you might want to reconsider using the server internal libraries. But for me generating invoices and such never is time-critical and usually done asynchronous using background tasks via shells :)

For DomPdf one important setting when using images in your PDFs is to “enable remote”:

define('DOMPDF_ENABLE_REMOTE', true);

Either way don’t forget the rest of the settings for CakePdf – it could look like this:

$config['CakePdf'] = array(
    'engine' => 'CakePdf.DomPdf',
    'options' => ...,
    'margin' => ...,
    'orientation' => 'portrait',
);

The official documentation on the installation part tells you all you need to know. Check it out.

Advanced Setup

The recommended way is to use CakePlugin::load('CakePdf', array('bootstrap' => true, 'routes' => true)); which is supposed to take care of automatic View class switching.

If you don’t want to load the plugin bootstrap/routes and rather do it manually you need to automatically map all .pdf extension URLs to the CakePdf plugin PdfView ourselves:

public $components = array(
    'RequestHandler' => array(
        'viewClassMap' => array('pdf' => 'CakePdf.Pdf')
    )
);

This can also be useful if you want to overwite/extend the PdfView class for some reason (MyCakePdf etc).

Skipping the plugin routes file is also useful, if you already have extension settings in your routes. You can then simply update your APP/Config/routes.php:

Router::parseExtensions();
Router::setExtensions(array('json', 'xml', 'rss', 'pdf')); // We just add pdf to the already defined ones

With both routes and bootstrap defined manually, CakePlugin::load('CakePdf'); suffices.

Usage

The official documentation already states that, so I won’t go into much detail. For the sake of completeness a short example:

public function view($id) {
    $this->pdfConfig = array(
        'filename' => 'invoice',
        'download' => (bool)$this->request->query('download')
    );
    $invoice = $this->Invoice->find('first', array('conditions' => array('id' => $id)));
    $this->set(compact('invoice');
}

If you use my Tools plugin, you can just use $invoice = $this->Invoice->get($id) here.

Make sure you have a basic pdf layout in /View/Layouts/pdf/default.ctp and a view template for the PDF in /View/Invoices/pdf/view.ctp

Now if you allow access without extension (/invoices/view/1) the action can render a normal HTML page with the invoice in a table like form.

If you link it as /invoices/view/1.pdf it will automatically switch the view class to PdfView and render it as PDF file:

$this->Html->link('View PDF', array(
        'action' => 'view', $id, 'ext' => 'pdf'
));

Note the trick with download using the query string for it. If you link to it with ?download=1 it will trigger the download instead of just displaying it:

$this->Html->link('Download PDF', array(
        'action' => 'view', $id, 'ext' => 'pdf', '?' => array('download' => 1)
));

This way the action can do both automatically.

Tips

URLs and Paths

For all engines to work with your URLs and images outputted by helpers and therefore with schema-less absolute URLs (/controller/action/ and /img/foo.jpg) this little modification in your AppHelper is quite useful:

/**
 * Overwrite to make URLs absolute for PDF content.
 *
 * @param mixed $url
 * @param bool $full
 * @return string
 */
public function url($url = null, $full = false) {
    if (!empty($this->request->params['ext']) && $this->request->params['ext'] === 'pdf') {
        $full = true;
    }
    return parent::url($url, $full);
}
 
/**
 * Overwrite to make paths for assets absolute so they can be found by the PDF engine.
 *
 * @param string $path
 * @param array $options
 * @return string
 */
public function assetUrl($path, $options = array()) {
    if (!empty($this->request->params['ext']) && $this->request->params['ext'] === 'pdf') {
            $options['fullBase'] = true;
    }
    return parent::assetUrl($path, $options);
}

This will make them absolute including the full base (schema + domain) in PDF context.

Filename and downloading

If you don’t force downloading and display the PDF, make sure you read the “eastereggs” part of serving-views-as-files-in-cake2/ regarding an additional passed param here to actually make the downloaded file name what you want it to be.

1 vote, 5.00 avg. rating (96% score)
 
No Comments

Posted by Mark in CakePHP

 

Using Composer

30 Mar

There are quite a few good articles on this topic, already. Especially on how to leverage it for CakePHP projects.

See

So I won’t go into detail too much. It is worth noting that composer really made a few things easier for us in the long run. And I was really against composer and packagist at the beginning as I didn’t want to use more dependencies than necessary. But that one sure is worth it if you use it right. And as every project pretty much uses it these days it is fairly stable and future proof to rely upon.

Installation

Easiest way: Download the phar and run php composer.phar [command]. You can just place this file into your app dir. You can also install it more globally, of course to just have to run composer [command] directly. The above articles or the composer docs explain that in more detail.

Usage

In general you can either use the composer commands to add dependencies or you can directly modify the composer.json file via editor (note that some IDEs support that natively with auto-completion). You also need to distinguish between “install” and “update” command. The first respects the current composer.lock file. The latter should only be used for development/staging, not for productive systems. See the basic usage docs for more details.

Composer and CakePHP

Can I switch my 2.x git project from submodules to composer?

Most definitly, yes. And you probably should. Using submodules really is a great annoyance, especially with the number of plugins increasing in a project. Unfortunately, getting rid of them is also not always easy – but doable.

You need to

  • git submodule deinit <path/to/plugin>
  • git rm <path/to/plugin>

Which, under windows, for example can proof a little bit difficult sometimes.

Then re-add them to the composer file and don’t forget to exlude the folder in the .gitignore file.

Composer and CakePHP3.x

In 3.x composer will pretty much be the de facto standard for CakePHP. So it is worth getting used to it now.

A few tips and tricks

I can add least talk about a few things I came across working with composer now for a few months.

VCS

Version control system repositories can easily be included if they don’t support composer or if you need to run a modified version of a 3rd party plugin/dependency. We then include the main source in the “require” section and add our specific (tweaked) fork as VCS in “repositories”:

"require": {
    "josegonzalez/cakephp-wysiwyg" : "dev-master",
},
"repositories" : [{
        "type" : "vcs",
        "url" : "https://github.com/MyOwnNameSpace/cakephp-wysiwyg" 
}],

Our version then contains some bug-fixes or additional stuff that needed it to branch off here.

dev prefix/suffix?

It kind of confused me when to use the suffix and when to use the prefix for “dev”. I found quite a look explanation from someone I added to this PR:

Basically dev releases are now simply branch names with a dev suffix for numeric branches which are comparable (3.0.*-dev) – or a dev prefix for textual names that are not comparable, like feature branches and master (dev-master)

You should authenticate the server/locally to avoid hitting the API limit

This page describes on how to generate an OAuth token and use this to avoid hitting the limit of requests per minute for GitHub.

Use proper user management to run composer commands

If you log in as root or any user that is not affialited with the www-data user, the files that are updated/created will most likely not be readable/usable. And using chmor -r 777 * sure is a really bad idea, as well. It is better to create a user that shares the same rights/group as www-data and use this one to deploy or do any changes on the code.

If something aint working

Try it again first – sometimes there are failures in the communication between GitHub, Packagist and your local connection.

Also try the self-update command to make sure you are running a recent version of composer. diagnose can also tell you of there is something wrong. Maybe you forgot to enable some required PHP extensions, like “curl” (which is necessary for it). But the error message usually tells you what exactly what the issue is.

0 votes, 0.00 avg. rating (0% score)
 
 

Continuous Integration with Jenkins

04 Mar

CI with Jenkins and GitHub is especially interesting for private repositories, as CI with Travis is mainly for free GitHub repos (unless you have Travis Pro, of course). But since Jenkins is OpenSource and free, it might make sense to set up this on your server and run CI directly in-house. It integrates nicely with GitHub. So the following tutorial will focus on that example.

So for the beginning we have our example app in GitHub, e.g. a CakePHP app (optionally including composer dependencies).

The main goals are:

  • Continuous test results on each push (especially to master).
  • Automatic test results for each PR (very important when working in teams) – linked inside the PR just as Travis would.
  • Coverage being analysed to see where the weak points can be (classes with < 50% test case coverage).

Set Up Jenkins

This is probably the most difficult part of all.

We first set up Jenkins.

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | apt-key add -
 
nano /etc/apt/sources.list
------------------------------
deb http://pkg.jenkins-ci.org/debian binary/
------------------------------
 
apt-get update
apt-get install jenkins

Check if Jenkins runs properly

http://hostname.domain.local:8080

Install plugins and all their dependencies

Let’s then include the plugins we need to integrate GitHub: “GitHub Pull Request Builder” and “GitHub plugin”. The first is especially interesting as it makes it possible to have Jenkins “build/pass” statuses directly in each PR (similar to Travis).

Access Data -> Global Access Data -> Add Access Data

Create access data for the repository you want to access. Provide username and password or keyfiles.

Manage Jenkins -> Configure System

  • Add your repositories under “Git”
  • Set “Manually manage hook URLs” under “GitHub Web Hook”
  • Generate token (https://github.com/settings/applications) with Github user which has access to the private repository and put the token under “GitHub pull requests builder”

Add Job

  1. Select “Free Style” project and provide a jenkins working space name
  2. Put your repository url (e.g. https://github.com/Name/repo) to “GitHub-Project” input field
  3. Select Git under “Source-Code-Management” Put your repository url again and select the access data from step 5 of this tutorial. Click on advanced settings and set “+refs/pull/:refs/remotes/origin/pr/” for Refspec and “origin” for name. Set also the branch specifier to “${sha1}”
  4. Select “Build when a change is pushed to GitHub” and “GitHub pull requests builder” under “Build trigger” Add persons or organizations to “GitHub pull requests builder” which are whitelisted for trigger an PR build request
  5. Add shell execution script for your test cases under build proceed (see below)
  6. Add “Set build status on GitHub commit” as post build action.

My shell execution script for the CakePHP apps looks like this:

cd /var/lib/jenkins/jobs/projectname/workspace/app
cp /var/www/projectname/app/Config/database.php Config/
mkdir -p tmp && chmod -R 777 tmp && rm -R tmp/*
cp /var/www/projectname/app/composer.phar ./ && php composer.phar update
Console/cake test app AllApp --stderr

It first navigates into the freshly pulled repo code and into the APP dir. Then it copies over the database.php from the current staging website (could also be some other place where you store them). It then creates tmp dirs and sets the correct permissions. The composer.phar file is also copied over and executed to pull all composer dependencies. At finally the AllApp (group) tests are executed – with –stderr in order for the session to work properly. One can then also apply -q for silent (=none) output.

Setup GitHub hooks

To trigger the build process on Jenkins, we need to configure service hook on GitHub. In order to achieve this, navigate to GitHub repository settings and configure Jenkins Hook URL for GitHub plugin. The URL format http://<jenkins-username>:<jenkins-password>@<Elastic-IP-Address>:8080/github-webhook/

Running tests

Run first test manually

Click on “Build with Parameters” and hit “Build” and you will see the first manually triggered build in progress. If all goes well the icon should be blue (pass) – not red (fail). In case you need to see what is going on, check the “Console Output” of a specific build.

Check automated tests

Commit to master and see if a build is triggered. Also open a PR and check if there is a badge displayed with the current build status.

Sugar

Badge

To have a small “build status: passing” badge in our Readme, we can install the plugin “Embeddable Build Status”. It will create a new menu entry from where you can select your badges including ready-to-use Markdown syntax if needed.

And if it is a private repo you might also want to htaccess the Jenkins web interface or protect it some other way from the public.

Code Coverage

You can install the plugin “Clover PHP Plugin”. Create a folder “coverage” in your workspace.

Add this to the above command:

--log-junit '../coverage/unitreport.xml' --coverage-html '../coverage' --coverage-clover '../coverage/coverage.xml'

And set up a “Publish Clover PHP Coverage Report” Post Build Action with the absolute path to this coverage.xml file in the workspace. In my case just coverage/coverage.xml as it can be relative to the workspace, as well. If you also want to publish the HTML coverage report (which is pretty neat), you can simply put coverage in there then.

Make sure you got Xdebug up and running (apt-get install php5-xdebug). It is a direct dependency here. Note that using code coverage reports will slow the whole process down by “x3″ at least. Usually that shouldn’t really make a difference, though.

In most cases you want to exclude few dirs. In case you are already using a phpunit.xml file which resides in your APP dir, you can easily add

<phpunit>
    <filter>
        <blacklist>
            <directory suffix=".php">../lib</directory>
            <directory suffix=".ctp">../lib</directory>
            <directory suffix=".php">*/Vendor</directory>
            <directory suffix=".php">*/tmp</directory>
            <directory suffix=".php">*/Config/Migration</directory>
            <directory suffix=".php">*/Config/Schema</directory>
            <directory suffix=".php">*/Console/Templates</directory>
            <directory suffix=".php">*/Test</directory>            
            <directory suffix=".php">*/Plugin</directory>
        </blacklist>
    </filter>
</phpunit>

You can also just create this file and also copy it over to the jenkins workspace.

Memory Limit

If you have a project of medium to large size you will soon see the memory usage for the test coverage of 50%+ way above 200MB. With more coverage and the project getter bigger, or some coverage report generated you will run out of memory with 256MB (which one would think should suffice). Raise your limit to at least 512 for the PHPUnit testing. You can use the phpunit.xml file:

<phpunit>
    <php>
        <ini name="memory_limit" value="512M" />
    </php>
    ...
</phpunit>

Cleanup

Use the plugin “Discard Old Build plugin” to keep your diskspace reasonable. Without it (especially with CodeCoverage) the disk space will soon exceed several GB. We usually keep the last 40 builds.

More

This article goes more into detail about the installation of Jenkins and also how to use additional tools like CodeSniffer, MessDetector and CO using a build file and Ant. But I didn’t check that out yet. There is also jenkins-php.org if you want to go all in :)

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in Testing