RSS
 

Migrating SVN apps to Git and Composer

23 Jul

Just recently I had to move a few remaining legacy CakePHP2.x apps from SVN to Git+Composer. This serves as a short how-to.

Note: Installing a new CakePHP 2.x app with Git and Composer right away is dead easy and explained quite well in the cookbook. This post is more about migrating existing ones.

SVN + Composer/Git?

They can actually work side-by-side. For a migration process this can be useful, especially when migrating huge apps. Then doing it piece by piece helps to avoid chaos.

For this little how-to the main repo will stay a SVN one until the very end for this very same reason.

First steps

We have our SVN repo – up to date thanks to svn update. Download composer.phar in your APP dir. This file should be added to your excludes. Set up a composer.json file in there, as well, and commit it.

CakePHP Core

You might have hard-coded the core files in ROOT/lib/Cake – or used an svn:externals on ROOT/lib. Either way, remove that and add CakePHP2.x as Composer dependency.

Update index.php and webroot.php

As per documentation, we now need this in those files:

define('CAKE_CORE_INCLUDE_PATH', ROOT . DS . APP_DIR . DS . 'Vendor' . DS . 'cakephp' . DS . 'cakephp' . DS . 'lib');

Resolve global vendors folder

In case you used a global vendors folder it will now not automatically be included anymore. To quick-fix this, add this to your bootstrap.php (at the beginning probably):

App::build(array('Vendor' => array(ROOT . DS . 'vendors' . DS)));

This will load your App::import()ed classes again from that folder.

In a second step you can then start moving those to the APP/Vendor folder – either as Composer dependency or hard-coded.

When all are transferred, you can remove the global vendors folder and also remove the additional App::build() call.

Resolve svn:externals for plugins

Most plugins will probably be hard-coded or included via svn:externals. Remove the svn:externals and add them as Composer dependency. Do the same for any plugins in your ROOT/plugins folder. If they are hard-coded there, move them to the APP/Plugin folder.

Almost done

At this point we already have a fully functional application again that loads all dependencies via composer – even though it is still a SVN repo. All it needs is

cd .../root && svn update
cd app && composer install

If some global vendor libs are harder to migrate or the “svn to git” conversation is troublesome, you can take your time get it up and running again on this level just fine. At some point you will have finished the migration though, then proceed to the final step.

Move SVN to Git

When all externals have been resolved, the root vendor and plugin folders have been cleaned and their content moved to the APP pendents, we can now start migrating the application to Git. For this we drop the usual /trunk/... folder structure and directly copy the APP folder from your SVN repo into the root of your new Git repo. The ROOT folder itself serves no purpose anymore, and we can have APP as root folder now. Therefore in root, you should now have the composer.json file. For an example, see cakephp-sandbox.

As for “copy”: You can either directly copy-and-paste your repo (quick and dirty), or use git-svn tools to preserve your changelogs/history. The latter is the cleaner approach, of course. See this guide on how to do that. Some services like GitHub might also offer a git-svn push access or link useful tools. Those approaches usually need a folder move of the APP folder content to the root level then on top.

Either way, the more you composer and the less you hard-code your libs, the quicker and easier the svn…git transformation will go.

Now updating app + core + plugins + vendors is just:

cd .../root
git pull
composer install

Since the APP folder is now the root folder, no need to further cd deeper.

Sugar

You may add

require_once dirname(__DIR__) . '/Vendor/autoload.php';

at the top of your bootstrap.php file to add autoloading via PSR for new namespaced repos (on top of the still working App::import() and App::uses() methods).

That means, that you can now use namespaces and third party packages via composer quite easily. Let’s say we want to use the MediaEmbed lib in our project. We then simply add it to our composer.json:

"require": {
    "dereuromark/media-embed": "dev-master"
}

In our CakePHP classes, e.g. an EmbedHelper, we can now use it right away:

<?php
use \MediaEmbed\MediaEmbed;
 
App::uses('AppHelper', 'View/Helper');
 
class EmbedHelper extends AppHelper {
    /**
     * @param string $url Absolute URL of the video/audio.
     * @return string|null HTML snippet or null on failure.
     */
    public function media($url) {
        $MediaEmbed = new MediaEmbed();
        if ($MediaObject = $MediaEmbed->parseUrl($url)) {
            return $MediaObject->getEmbedCode();
        }
    }
}

Isn’t that awesome? You now have the full PHP5.3+ (GitHub) lib variety out there at your disposal. You might want to check out this list of cool libs.

Private repos

With the externals, it was quite easy to pull from your own repo URLs. Composer makes this a little bit more difficult. To overcome the issue for GitHub private repos, for example, you can use vcs:

"require": {
    ...
    "your-username/cakephp-some-private-plugin" : "*"
},
"repositories" : [{
        "type" : "vcs",
        "url" : "git@github.com:your-username/cakephp-some-private-plugin"
    },
    ...
],

Note that this will prompt once for your username and password and create a token for re-use:

Your GitHub credentials are required to fetch private repository metadata. The credentials will be swapped for an OAuth token stored in …/Composer/auth.json, your password will not be stored.

Last tips

This tutorial assumes that you are properly using the current master (2.5.3 at this time) version of CakePHP. Always upgrade to the current master – never use anything older.

Make sure you have a current index.php/test.php file as well as a current APP/Console dir including those cake scripts (current as in master/2.5.3 at least). You can just copy them over from the CakePHP source folder (Cake/Console/Templates/...) if you are unsure. Especially for console this is quite important. Otherwise Console/cake might not work out of the box with the new composer approach – even though it would if you had up-to-date files.

Also, don’t forget to .gitignore all included dependencies (both Vendor and Plugin ones). And also the ones from SVN, composer.phar etc.

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in CakePHP

 

HTML, Markdown or BBCode?

16 Jul

Or something different? Or just plain text maybe?

“User input is hard”

There are pretty prejudiced blog posts in favor or against the usage of BBCode for blog posts, comments etc. Here for instance – continuing on this second post.

Personally, I am in favor of some abstraction for the frontend user, usually BBCode or a similar syntax. But it always depends on the usage in the app. For forums, blog post comments, etc they are sure quite useful and avoids exposing HTML to users.

Let’s dig into some options more deeply.

Plain text

We all probably do or did it: Using .txt files to store some pseudomarkup enriched text including lists and headings. While its simple and easy, the output on a a browser for such textual representations is pretty poor. It starts with missing <p> paragraphs or newlines. All those would have to be manually modified. In CakePHP there is a helper method that does exactly that: TextHelper::autoParagraph(). Adding autoLink() you even get your URLs and emails enriched as HTML. It works perfectly for all varchar or text DB fields that are exposed as textareas in the forms and only needs to accept plain text. This is all nice for very basic use cases, but is quite limited in the presentation.

HTML and WYSIWYG Editors

This widely used in backends as this is easy to implement. Just add a JS based WYSIWYG editor on top of your textareas and you got yourself editing in the way it will be presented upon output. This live preview while typing sure is a nice bonus here. One drawback often is the amount of overhead added via tags, classes and alike. But that depends on the editor being used.

Often times there is a “plain HTML” button to even allow custom HTML modifications in the source code. While this is probably the most flexible approach, it is also the most dangerous. You can easily screw up the HTML – invalid HTML is hard to spot and might break your whole frontend layout.

Tools such as htmlpurifier exist to support those editors in removing unwanted content, fixing broken HTML and cleaning up the source code (mess). This also makes it possible to allow a subset of HTML in non-backend textareas untrusted users can have access to. In CakePHP, for example, you can attach a PurifiableBehavior to your model that cleans the content upon saving.

The most commonly used editors are probably listed on this comparison site.

BBCode

That was the very first abstraction level available. A lot of forum software still uses it quite thoroughly.

Everybody who opposes BBCode completely (like the above posts), does not know much about the user perspective. Users don’t care about semantics and just want their link to be added. They want it simple and straight forward. So [img]url[/img] in BBCode would be more intuitive as the resulting <img src="url" /> in HTML. Now what is easier to understand for a newbie? What is easier to read? Of course we have a little bit more processing with BBCode. But with caching that is minimized to nothing. For admin backends it is usually easier to use HTML. This way they have more tags, attributes to chose from. We can also assume that they don’t want to harm the site and that they know what they’re doing.

The second important point is abstraction. [code] can be <code>... or even <pre>... or a combination of both. Everybody understands [code] whereas [pre] etc is not so understandable. So we use [code] and afterwards transform it into the more complex HTML tags we need for markup. But the user text stays clean and straight-forward. He doesn't need to know about the mapping of [code=php] to <pre id="xyz" class="php"><code>...

jbbcode looks like a solid implementation for this.

There is even a WSYIWYG editor for this now. Even though that one is based on JS - meaning you would have to keep your custom rules redundant, once in PHP and once in JS for this editor. Also, there might be slight edge case differences between preview and actual result.

Markdown

This is becoming more and more popular these days. Not only for developers who use GitHub a lot, but also by bloggers using WordPress plugins for this or other websites that want to avoid the HTML overhead when displaying lightweight markup text. The benefit here is that the text can be written almost as normal text. And even non-developers would easily understand lists such as:

- one
- two
- three

So it combines text with leightweight markup that is easily understandable by everyone - and probably even used intuitively without knowing it. Translation into HTML is straightforward.

Currently most people prefer the GitHub flavored addons as the original markdown implementation hasn't had any progress anymore the last years. A nice demo and comparison shows the difference.

A slower but probably more powerful library is Ciconia. It is intended to be more flexible and extendable.

There are nice WSYIWYG editor implementations for Markdown: sofish.github.io/pen or markitup. But as with BBCode: since that one is based on JS - meaning you would have to keep your custom rules redundant again, once in PHP and once in JS for this editor. Also, there might be slight edge case differences between preview and actual result.

Imagine you can write all your HTML in such a DRY and non-HTML-polluted way and still get nice HTML from it. And you can also use it for textual representation right way (e.g. text emails). Awesome.

Speaking of - there are nice tools that can actually take your already written HTML and revert it back into Markdown. See http://blog.oddbit.com/2012/11/06/convert-html-to-markdown/ and to-markdown.

So if you have already existing records or blog posts in the old format, you might be able to convert them and then use markdown only from there on.

Summary

From what we read so far, the complexity would probably be best described as:

Text < Markdown < BBcode < HTML

And the further left, the better - not only for interoperability. You should use the easiest format that suits your need.

HTML

  • No additional parsing needed (once it is validated and saved)
  • Cannot work without a sanitizing process for non-admins (stripping off any unwanted attributes or unsafe elements)

BBCode

  • Simpler sanitizing
  • Abstraction possible ([ video ] or [spoiler] tag)
  • Does not interfere with HTML Markup (e.g. for code snippet posts in dev blogs)

Markdown

  • A compromise between writing plain text and using minimal additional markup to enhance it
  • Intuitive
  • Less error-prone than BBCode regarding simple tags/markup
  • Maybe more error-prone than BBCode for more complex tags/markup
  • Nice for inline references/images/hyperlinks (links can be grouped at the bottom)

Combining them

Sometimes, the lightweight Markdown might not cover everything. Writing a custom wrapper you can easily combine them enhancing your toolset.

You use Markdown as primary parser and parse the remaining BBCodes afterwords. HTML could be allowed using a custom markdown rule, e.g.:

```html
<myhtml />
```

You could also use BBCode then, of course:

[html]
<myhtml />
[/html]

Adding plain HTML in between the Markdown and BBCode markup would work, as well (I do that^^). But this can easily break or have unseen side effects when trying to escape the source code. It is more difficult to distinguish between an HTML tag <b>I am bold</b> and just plain text containing those chars by accident: I like the brand <FooBar>. In that case all those non-HTML-intended characters need to be properly escaped, which really is annoying. So please don't do that ;)

In the example above the whole page would be escaped using h() (the htmlspecialchars() in CakePHP). This way it is secure by default. And the tags from above would automatically undo h() to display the raw HTML again.

I think combining them in a logical order can in some cases make all the difference and solve all your problems at once. You have the simple and lightweight markup as basis, you are able to apply custom codes via BBCode similar syntaxes and on top you can always use real HTML for more complex scenarios (tables and alike) where necessary.

Side notes

This blog also uses Markdown for all posts and some BBCode for the comments, of course.

I did not mention Textile as even though introduced shortly before Markdown it never really became that popular. Those are similar, though, in their ideas. A full list of further lightweight markup languages can be found at wikipedia.org.

Further addons

I use anchorjs to automatically add anchors on the fly using JS. This is especially useful if the parsed markdown itself produces headings without any attributes. This way they are added without having to dig deeper into the post-processing of the markdown parser or modifying the resulting HTML. Another alternative would be anchorific.

You can use my MediaEmbed lib as addon for BBCode or Markdown (or even plain text) to auto embed Video snippets. See the examples/bbcode.php there for a live example.

Further Links and Resources

This BBCode parser once looked quite promising. But it now seems abandoned.

The MarkupParsers Plugin combines several markup syntaxes into a plugin. There are even MarkdownView classes which would render a complete markdown-flavored layout into HTML. For me a helper or lib wrapper usually suffices as I usually only output parts of the layout as such a markup-flavored text.

See dillinger.io/ or stackedit.io for editing Markdown in real time. And there is a cheat sheet to go along with it.

Outview

I will probably add some real life examples and comparisons soon on my sandbox site. Stay tuned.

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in HTML, WebDevelopment

 

CakePHP 2.5 arrived + CakePHP Tips 2014

13 May

You probably read my last tip sections. And I started to move some of them to my sandbox app.

But once in a while it might also be nice to publish a few selected tips, here as well.

Oh, and CakePHP 2.5 is out! Get it – now :) See the last chapter of this post about why you really should upgrade ASAP. And it doesn’t matter if you are on 1.x or 2.x.

URLs in CLI

In CLI and your shells there is no HTTP HOST and therefore usually no absolute URLs. But in order to properly work with links/urls (e.g. sending batch emails), we need that to work. So I use Configure::read(‘Config.fullPageUrl’) here. My configs.php file then contains:

$config['Config'] = array(
    'fullPageUrl' =>  http://www.myapp.de // The primary URL
)

In case you have a different domain for local development like http://myapp.local and you want that to be the fullPageUrl, make sure you overwrite the default in your configs_private.php file:

Configure::write('Config.fullPageUrl', 'http://myapp.local'); // Usually defaults to the (live) primary URL

And in the end of my bootstrap file (after including all config files), I simply do:

if (php_sapi_name() === 'cli') {
    Configure::write('App.fullBaseUrl', Configure::read('Config.fullPageUrl'));
}

To test the current domain/fullBaseUrl, you can use my Setup plugin command cake Setup.TestCli router. It will output both relative and absolute URL examples generated by the Router class with your current settings.

So on the live server then it will output http://myapp.local instead of http://localhost when generating Router::url()s in your shells.

Careful with running shells in CLI

Most are probably not aware, but running shells in CLI needs to have a proper user management around them in most cases. Imagine yourself running your apache as www-data (default) and log in as root or any other user not affiliated with that www-data user/role (bad idea). Once you execute a shell and tmp cache data are (re)built, your www-data user cannot access them anymore, losing the ability to cache and triggering a lot of errors. So make sure you only log in with a user that shares the role of www-data at least, so that both can access each others’ generated files.

A popular example is the ClearCache shell which re-builds your cache dirs in debug 0 (when changing files or db schema makes this necessary).

PS: Of course you could also switch to another cache system than the default File engine. But most probably didn’t do that yet, either.

Merging arrays

Ever wondered what Hash::merge(), array_merge and array_merge_recursive have in common – or don’t have in common? Check out these merge comparison examples.

See what the requirements are – and use the appropriate merge method then.

There is also the + operator, which is quite useful when merging flat arrays and string key based options. This is quite commonly used in the core to merge options and defaults:

$defaults = array(
    'escape' => true
):
$options += $defaults;

In this case the $defaults are merged on top of $options, but only if that key has not been specified yet. This kind of merge is really efficient and fast (4-5x faster than array_merge() itself) – but should really only be used if all keys are definitely strings.

Paginating and merging

A propos merging: When setting up paginate settings in your controllers, try to prevent

public function index() {
    $this->paginate = array(...);
}

This way you kill all defaults you might have set via public $paginate or in your extending controllers (AppController’s beforeFilter() callback for example).

So it is better to use:

$this->paginate = array_merge($this->paginate, array(...));
// or
$this->paginate = array(...) + $this->paginate;

In my 2.x code for example I have this snippet in all my AppControllers to have query strings for paginations:

public function beforeFilter() {
    parent::beforeFilter();
 
    $this->paginate['paramType'] = 'querystring';
}

This will only work with proper merging of defaults and custom settings. I prefer the latter because the settings are string based and here the + operator is the fastest and easiest way of doing things. Once the key is already set in your method, the default will be ignored right away (with array_merge() and nullish values this can be different/unexpected).

And remember to not mix controller and component pagination.

Pagination and sort default order

Adjust your bake templates so that some fields like created/modified are ordered DESC instead of ASC per default in the index actions. For those fields the first click on the header row should display them DESC right away as one would then most likely be interested in the latest changes. Same goes for most date fields like “published” as well as fields like “priority”, “rating”, …

That’s how the baked code (or manually adjusted one if done afterwards) could then look like (index.ctp):

<th><?php echo $this->Paginator->sort('name');?></th>
<th><?php echo $this->Paginator->sort('amount');?></th>
<th><?php echo $this->Paginator->sort('priority', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('status', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('publish_date', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('modified', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('created', null, array('direction' => 'desc'));?></th>

Using modified model data in the form again

Some of you might have had the wish of posted data that was modified in the model due to beforeValidate/beforeSave to appear modified in the view again (so that the reason for validation errors might be more clear etc). So let’s say you have a beforeValidate callback to clean the input of a textarea:

public function beforeValidate($options = array() {
    if (isset($this->data[$this->alias]['comment']) {
        $this->data[$this->alias]['comment'] = $this->_clean($this->data[$this->alias]['comment']);
    }
    return true;
}

So in this case it could easily be that _clean() removes some invalid content and thus the minLength rule is suddenly triggered. Which is weird, since we posted at least twice the length of text. To clarify to the user what is going on, one could adjust the error message – but one could additionally return the modified (ready to save) data instead of the actually posted data.

public function add() {
    if ($this->request->is('post')) {
        $this->{$this->modelClass}->create();
        if ($this->{$this->modelClass}->save($this->request->data)) {
            $this->Session->setFlash(...);
            return $this->redirect(array('action' => 'index'));
        } else {
            // Here we assign the modified model data back to the request
            // object and therefore to the view/form
            $this->request->data = $this->{$this->modelClass}->data;
            $this->Session->setFlash(...);
        }
    } else {
        // Default values for the form
        $this->request->data[$this->modelClass]['status'] = true;
    }
 
}

The input field will now contain the content that was served to beforeValidate(). And combined with a good error message this will probably clear things up.

PS: You can also directly use the model’s name instead of {$this->modelClass}, e.g. ‘Comment’.

MySQL – MyISAM vs InnoDB

InnoDB is a little bit more robust as it allows transactions. Especially with CakePHP and “multiple queries” most of the time (per default) this can be quite helpful in keeping the DB in a valid state. Also read myisam-versus-innodb on pros/cons for each.

One additional problem with InnoDB, though: Per default it creates a file that always increases in size, never decreases again. This can soon be a nightmare with backuping when it becomes >> xx GB of size. See how-to-shrink-purge-ibdata1-file-in-mysql how to avoid that by not using that file, and instead using innodb_file_per_table.

Testing

Testing Controllers

I stumbled upon a few issues with testing controllers – especially plugin controllers. For plugin controllers to be testable unfortunately you always need to use $uses, even if it wasn’t necessary due to conventions. So for your Tools.TinyUrls Controller you would need

public $uses = array('Tools.TinyUrl');

Otherwise it would always try to look for the model/fixture in your app dir, which eventually always fails.

Do not forget to use --stderr when testing in CLI. Otherwise all tests that use the session (and which work fine in webtest runner) will fail:

cake test app AllController --stderr

Test Coverage

If you want to generate a HTML overview of all your locale test coverage:

cake test app AllApp --stderr --log-junit tmp/coverage/unitreport.xml --coverage-html tmp/coverage --coverage-clover tmp/coverage/coverage.xml

The report index.html will be in your /tmp/coverage folder.

More on testing – especially controller testing – can be found on the corresponding Tools plugin Wiki page.

Upgrade (deprecated) code

It is always good practice to upgrade to the current master. Not only for CakePHP, but there especially. It will save you a lot of time in the long run, as migration will be easier and faster in small steps instead of one huge step. It will also make it easier to use the new features and more importantly will also come with a lot of fixes and method corrections that the older versions didn’t get anymore. Those outdated versions usually only get security-bugs fixed. So if you look hours for an error that is already fixed in the current master, it was just a huge waste of time. I have seen that a thousands times – on IRC and elsewhere.

So in case you are not using the current master (2.5), do it now. Internally, upgrading 2.x is a “piece of cake”. Upgrading from 1.x is also not that big of a deal – just needs a little bit more manual adjustments. For most things you can use the core UpgradeShell as well as my Upgrade plugin.

In case you are already upgraded to 2.5, you can and should also remove deprecated functionality in favor of the right one. Those deprecated things will only add to the file of upgrades for the next 2.x release or 3.x. And using the upgrade shell it is usually just one single command to execute. So for 2.5, you should have removed all the “easy” stuff that will clearly be switched with a different way of doing things as it is mentioned in the migration guide, e.g.

  • loggedIn() in favor of Auth::user(‘id’)
  • CakeRequest::onlyAllow() in favor of CakeRequest::allowMethod()
  • Use first argument (string|array) instead of var args for CakeRequest::allowMethod(), AuthComponent::allow(), etc
  • $title_for_layout in favor of $this->fetch(‘title’) and all other deprecated view vars

From 2.4 and below there are also a few left overs that could easily be corrected:

  • FULL_BASE_URL, DEFAULT_LANGUAGE, IMAGES_URL, JS_URL, CSS_URL to Config variables
  • Remove CAKEPHP_SHELL, IMAGES, JS, CSS usage
  • Simplify HtmlHelper::css()
  • Remove Sanitize class usage in favor of a more sane approach
  • Simplify CakeRequest and PaginatorHelper usage with param() if applicable
  • Don’t use request shortcuts anymore – like $this->action in favor of $this->request->action etc
  • Get rid of Model::read() calls – in 3.x this will be get() – I use my Tools plugin MyModel::get() method here already for years
  • Use the new Model::save() syntax for options
  • Completey get rid of named params in favor query strings (Use my Tools plugin CommonComponent and Configure value App.warnAboutNamedParams to detect left-overs)
  • Replace all Set calls with Hash calls, as Set is deprecated. Make sure it is tested thoroughly (as the functionality of Hash might be slightly different).
  • Prevent using Validation::range() in favor of a custom validation rule for inclusive ranges to easier migrate to 3.x – or simply use my Tools plugin MyModel::rangeInclusive() method.
  • Further deprecations in favor of the 2.5+ way to do things

and so on. For some details see Tips-Upgrading-to-CakePHP-2.x.

My Tools plugin also contains a few further tweaks that can help ease migration from 2.x to 3.x See the full list on the corresponding Wiki page.

This will help making sure any future upgrade is less painful. Think about it: When you do that now the remaining TODO list will only be half the size and look a lot less intimidating. When the time comes to upgrade to 3.x it will look quite more doable.

My codez is now officially all 2.5 – and as 3.0 ready as it can get :P

And what always helps a lot is to code clean and structured. A code mess will always be difficult to maintain. So use coding standards and enforce them. Use best practice approaches. Refactor once in a while to prevent a mess from happening. Happy coding :)

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in CakePHP

 

CakePHP flash messages 2.0

21 Apr

This is a topic often discussed. What is the best approach? While I might not have the best, I sure have a fully working one since 2008 and CakePHP1.3. I still use it in all 2.x apps. It is easy and does not require a lot of overhead.

Basically, it enhances the core one with the following features

  • Different types possible (now in 2.x kind of in the core, as well, using elements)
  • Multiple flash messages per type possible (with a max limit to avoid session flushing)
  • Transient messages (via Configure) and real ones (via Session)
  • Transient ones can also be fired from the views (to display some hint/info for this page) – although one should try to stick to the controller

Demo

sandbox/examples/messages

How does it work?

We attach the Common Component and the Common Helper to the AppController:

public $components = array(..., 'Tools.Common');
public $helpers = array(..., 'Tools.Common');

In our layouts we need to switch the default output to ours:

echo $this->Common->flash();

And we can use `em:

public function add() {
    ...
    if ($this->Entry->save()) {
        $this->Common->flashMessage('The Entry has been saved.', 'success');
    } else {
        $this->Common->flashMessage('The Entry could not be saved. Please check the form.', 'error');
    }
}

Styling

We style our flashmessages via CSS

div.flash-messages {
    width: 90%;
}
div.flash-messages div {
    padding: 10px;
    padding-left:40px;
}
div.flash-messages div.error {
    width:100%; border-style: solid; border-width:1px; border-color:#B84D17; 
    margin-right:2px; color:#000000; margin-bottom:8px;
    background: url(/img/css/layout/icon_error.gif) left center no-repeat;
    background-color:#F7C6A5;
}
div.flash-messages div.warning {
    width:100%; border-style: solid; border-width:1px; border-color:#D0C130; 
    margin-right:2px; color:#000000; margin-bottom:8px;
    background: url(/img/css/layout/icon_warning.gif) left center no-repeat;
    background-color:#F6F3A4;
}
div.flash-messages div.success {
    width:100%; border-style: solid; border-width:1px; border-color:#009900; 
    margin-right:2px; color:#000000; margin-bottom:8px;
    background: url(/img/css/layout/icon_success.gif) left center no-repeat;
    background-color:#A5F7A8;
}
div.flash-messages div.info {
    width:100%; border-style: solid; border-width:1px; border-color:#cccccc; 
    margin-right:2px; color:#000000; margin-bottom:8px;
    background: url(/img/css/layout/icon_info.gif) left center no-repeat;
    background-color:#ffffff;
}

Or any other layout for that matter. The images are small icons and also optional, of course :)

Transient flash messages

You can also put flash messages on top that are not stored in session but Configure (for this request only). This can be useful if you don’t intend to redirect and don’t want them to show up if that happens.

From your controller:

$this->Common->transientFlashMessage('This page is currently being redesigned', 'info');

Or from your view ctp (even elements, blocks or the layout if it happens prior to the flash message output):

$this->Common->addFlashMessage('This page is under maintenance. It may be broken!', 'warning');

Output in a specific order or only specific types

You can filter the output, both in order and types:

// Using Common helper
echo $this->Common->flash(array('warning', 'error'));

In this case it would only output the warning and error messages, in this order (Usually the order is “error, warning, success, info”).

Details

For details please the Wiki page.

Outview

It would probably be nice to add element support at some point. This would allow an easier approach to customization of those messages.

There are also quite a few CakePHP core tickets open regarding flash message enhancements – see this or this which might even lead to an own component for it some day. But until then my approach will still be used in all my xx apps :) So in 3.x there will be a Flash component and Flash helper to provide a clean way to produce flash messages. If they provide the same features my implementation currently does, will have to be investigated. But it will use templating which will sure be nice.

0 votes, 0.00 avg. rating (0% score)
 
3 Comments

Posted by Mark in CakePHP

 

Update your Server to PHP5.4+

13 Apr

With many new features and more speed it makes sense to upgrade to PHP5.4. It is also a requirement when you start using CakePHP3.0 (the dev releases) or code that contains short array syntax. So the sooner you upgrade the better.

The following examples are for Debian Squeeze.

Backup

Make sure you have a backup of your server/files/db and your php.ini and other config files.

Updating

Debian doesn’t offer this natively. We need to add some special packages for this.

Open the sources list using nano, vi or the edit tool of your choice:

vi /etc/apt/sources.list

And add those two lines at the end:

deb http://packages.dotdeb.org stable all
deb-src http://packages.dotdeb.org stable all
 
deb http://packages.dotdeb.org squeeze-php54 all
deb-src http://packages.dotdeb.org squeeze-php54 all

We need a public key to avoid those warnings showing up:

wget http://www.dotdeb.org/dotdeb.gpg
cat dotdeb.gpg | apt-key add -

Use apt-get update and apt-get install php5.

That should take care of it. PHP5.4.x should now be installed and ready to work with. And have fun playing around with CakePHP3.0 or some new features of PHP5.4 :)

Keep other packages up to date

Also make sure other packages are up to date using apt-get upgrade.

If it doesn’t install right away, one might also need apt-get dist-upgrade. Careful with this, though. It might install/overwrite more than it should.

Dev Libraries

If php5-dev libraries are needed, they can be installed via apt-get install php5-dev.

Key Benefits of 5.4

  • Speed/Stability
  • Traits
  • Closures and $this
  • callable as typehint
  • Short array syntax (no mixin of method brackets and array brackets anymore)

The only real benefit coming in 5.5 is “Generators” which will allow handling of huge arrays and larger amount of data more gracefully. But other than that 5.4 is just fine for the moment.

Features of 5.3 you can now also use in Cake3

  • Namespaces
  • Closures
  • Late Static Bindings
  • Ternary shorthand form
  • Magic method __callStatic()
  • Nested exceptions
  • These features weren’t used in Cake2 core (and probably most apps) so far due to the BC for PHP5.2. If you already updated your minimum requirement for your server and application, those are also now of great use.

Further tips

The encoding even in more current verions of apache usually defaults to the old “ISO-8859-1″ one. You should make it “utf-8″ per default in your php.ini:

; PHP's default character set is set to empty.
; http://php.net/default-charset
default_charset = "utf-8"

Then even error pages or non-normal responses will be properly displayed. It is sad that even 2014 and in PHP5.4 the default is still not a proper one – probably due to BC or something.

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in PHP

 

Generating PDFs with CakePHP

08 Apr

The recommended approach is to use the CakePHP plugin CakePdf for this. With that plugin it is “a piece of cake” to output HTML views as PDF or render them to files for download/emailing etc.

You can obviously just use an approach just like this article describes. But this makes the code quite dependent on the actual rendering library used isn’t very DRY when there is more than a single page that needs PDF rendering.

Note: This article is about generating PDF files from HTML templates. If you already have PDF files you just want to output or serve as download, then this article is not for you. See serving-views-as-files-in-cake2 then.

Demo

A live demo can be found in my sandbox app. It also helps to compare the different engines.

Installation

First you need to decide on what engine you want to go with.

WkHtmlToPdf is the fastest, but it also depends on CLI and the binaries being installed properly. So it might work differently or not at all on different OS. But using binary files it should work out of the both on both UNIX and Windows. The latter just needs a single exe file drag-and-dropped on your system. wkhtmltopdf.org/downloads.html contains such a ready-to-use binary. You can then link it in your Configure set-up for CakePdf:

Configure::write('CakePdf.binary', YOUR_PATH_TO . 'wkhtmltopdf\wkhtmltopdf.exe');

For UNIX those precombiled binaries on that download page might not work and a simple apt-get install wkhtmltopdf usually won’t do the trick. You need to manually compile it in this case. But on most systems it should be fine. Don’t forget to adjust the path to the binary here, as well, if it is not the default one:

Configure::write('CakePdf.binary', YOUR_PATH_TO . 'wkhtmltopdf/bin/wkhtmltopdf');

I usually go with DomPdf as it is quite reliable, and (as PHP code) even though slower than a native approach will work flawlessly with all OS and setups without additional dependencies. If speed really is an issue you might want to reconsider using the server internal libraries. But for me generating invoices and such never is time-critical and usually done asynchronous using background tasks via shells :)

For DomPdf one important setting when using images in your PDFs is to “enable remote”:

define('DOMPDF_ENABLE_REMOTE', true);

Either way don’t forget the rest of the settings for CakePdf – it could look like this:

$config['CakePdf'] = array(
    'engine' => 'CakePdf.DomPdf',
    'options' => ...,
    'margin' => ...,
    'orientation' => 'portrait',
);

The official documentation on the installation part tells you all you need to know. Check it out.

Advanced Setup

The recommended way is to use CakePlugin::load('CakePdf', array('bootstrap' => true, 'routes' => true)); which is supposed to take care of automatic View class switching.

If you don’t want to load the plugin bootstrap/routes and rather do it manually you need to automatically map all .pdf extension URLs to the CakePdf plugin PdfView ourselves:

public $components = array(
    'RequestHandler' => array(
        'viewClassMap' => array('pdf' => 'CakePdf.Pdf')
    )
);

This can also be useful if you want to overwite/extend the PdfView class for some reason (MyCakePdf etc).

Skipping the plugin routes file is also useful, if you already have extension settings in your routes. You can then simply update your APP/Config/routes.php:

Router::parseExtensions();
Router::setExtensions(array('json', 'xml', 'rss', 'pdf')); // We just add pdf to the already defined ones

With both routes and bootstrap defined manually, CakePlugin::load('CakePdf'); suffices.

Usage

The official documentation already states that, so I won’t go into much detail. For the sake of completeness a short example:

public function view($id) {
    $this->pdfConfig = array(
        'filename' => 'invoice',
        'download' => (bool)$this->request->query('download')
    );
    $invoice = $this->Invoice->find('first', array('conditions' => array('id' => $id)));
    $this->set(compact('invoice');
}

If you use my Tools plugin, you can just use $invoice = $this->Invoice->get($id) here.

Make sure you have a basic pdf layout in /View/Layouts/pdf/default.ctp and a view template for the PDF in /View/Invoices/pdf/view.ctp

Now if you allow access without extension (/invoices/view/1) the action can render a normal HTML page with the invoice in a table like form.

If you link it as /invoices/view/1.pdf it will automatically switch the view class to PdfView and render it as PDF file:

$this->Html->link('View PDF', array(
        'action' => 'view', $id, 'ext' => 'pdf'
));

Note the trick with download using the query string for it. If you link to it with ?download=1 it will trigger the download instead of just displaying it:

$this->Html->link('Download PDF', array(
        'action' => 'view', $id, 'ext' => 'pdf', '?' => array('download' => 1)
));

This way the action can do both automatically.

Tips

URLs and Paths

For all engines to work with your URLs and images outputted by helpers and therefore with schema-less absolute URLs (/controller/action/ and /img/foo.jpg) this little modification in your AppHelper is quite useful:

/**
 * Overwrite to make URLs absolute for PDF content.
 *
 * @param mixed $url
 * @param bool $full
 * @return string
 */
public function url($url = null, $full = false) {
    if (!empty($this->request->params['ext']) && $this->request->params['ext'] === 'pdf') {
        $full = true;
    }
    return parent::url($url, $full);
}
 
/**
 * Overwrite to make paths for assets absolute so they can be found by the PDF engine.
 *
 * @param string $path
 * @param array $options
 * @return string
 */
public function assetUrl($path, $options = array()) {
    if (!empty($this->request->params['ext']) && $this->request->params['ext'] === 'pdf') {
            $options['fullBase'] = true;
    }
    return parent::assetUrl($path, $options);
}

This will make them absolute including the full base (schema + domain) in PDF context.

Filename and downloading

If you don’t force downloading and display the PDF, make sure you read the “eastereggs” part of serving-views-as-files-in-cake2/ regarding an additional passed param here to actually make the downloaded file name what you want it to be.

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in CakePHP

 

Using Composer

30 Mar

There are quite a few good articles on this topic, already. Especially on how to leverage it for CakePHP projects.

See

So I won’t go into detail too much. It is worth noting that composer really made a few things easier for us in the long run. And I was really against composer and packagist at the beginning as I didn’t want to use more dependencies than necessary. But that one sure is worth it if you use it right. And as every project pretty much uses it these days it is fairly stable and future proof to rely upon.

Installation

Easiest way: Download the phar and run php composer.phar [command]. You can just place this file into your app dir. You can also install it more globally, of course to just have to run composer [command] directly. The above articles or the composer docs explain that in more detail.

Usage

In general you can either use the composer commands to add dependencies or you can directly modify the composer.json file via editor (note that some IDEs support that natively with auto-completion). You also need to distinguish between “install” and “update” command. The first respects the current composer.lock file. The latter should only be used for development/staging, not for productive systems. See the basic usage docs for more details.

Composer and CakePHP

Can I switch my 2.x git project from submodules to composer?

Most definitly, yes. And you probably should. Using submodules really is a great annoyance, especially with the number of plugins increasing in a project. Unfortunately, getting rid of them is also not always easy – but doable.

You need to

  • git submodule deinit <path/to/plugin>
  • git rm <path/to/plugin>

Which, under windows, for example can proof a little bit difficult sometimes.

Then re-add them to the composer file and don’t forget to exlude the folder in the .gitignore file.

Composer and CakePHP3.x

In 3.x composer will pretty much be the de facto standard for CakePHP. So it is worth getting used to it now.

A few tips and tricks

I can add least talk about a few things I came across working with composer now for a few months.

VCS

Version control system repositories can easily be included if they don’t support composer or if you need to run a modified version of a 3rd party plugin/dependency. We then include the main source in the “require” section and add our specific (tweaked) fork as VCS in “repositories”:

"require": {
    "josegonzalez/cakephp-wysiwyg" : "dev-master",
},
"repositories" : [{
        "type" : "vcs",
        "url" : "https://github.com/MyOwnNameSpace/cakephp-wysiwyg" 
}],

Our version then contains some bug-fixes or additional stuff that needed it to branch off here.

dev prefix/suffix?

It kind of confused me when to use the suffix and when to use the prefix for “dev”. I found quite a look explanation from someone I added to this PR:

Basically dev releases are now simply branch names with a dev suffix for numeric branches which are comparable (3.0.*-dev) – or a dev prefix for textual names that are not comparable, like feature branches and master (dev-master)

You should authenticate the server/locally to avoid hitting the API limit

This page describes on how to generate an OAuth token and use this to avoid hitting the limit of requests per minute for GitHub.

Use proper user management to run composer commands

If you log in as root or any user that is not affialited with the www-data user, the files that are updated/created will most likely not be readable/usable. And using chmor -r 777 * sure is a really bad idea, as well. It is better to create a user that shares the same rights/group as www-data and use this one to deploy or do any changes on the code.

If something aint working

Try it again first – sometimes there are failures in the communication between GitHub, Packagist and your local connection.

Also try the self-update command to make sure you are running a recent version of composer. diagnose can also tell you of there is something wrong. Maybe you forgot to enable some required PHP extensions, like “curl” (which is necessary for it). But the error message usually tells you what exactly what the issue is.

0 votes, 0.00 avg. rating (0% score)
 
 

Continuous Integration with Jenkins

04 Mar

CI with Jenkins and GitHub is especially interesting for private repositories, as CI with Travis is mainly for free GitHub repos (unless you have Travis Pro, of course). But since Jenkins is OpenSource and free, it might make sense to set up this on your server and run CI directly in-house. It integrates nicely with GitHub. So the following tutorial will focus on that example.

So for the beginning we have our example app in GitHub, e.g. a CakePHP app (optionally including composer dependencies).

The main goals are:

  • Continuous test results on each push (especially to master).
  • Automatic test results for each PR (very important when working in teams) – linked inside the PR just as Travis would.
  • Coverage being analysed to see where the weak points can be (classes with < 50% test case coverage).

Set Up Jenkins

This is probably the most difficult part of all.

We first set up Jenkins.

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | apt-key add -
 
nano /etc/apt/sources.list
------------------------------
deb http://pkg.jenkins-ci.org/debian binary/
------------------------------
 
apt-get update
apt-get install jenkins

Check if Jenkins runs properly

http://hostname.domain.local:8080

Install plugins and all their dependencies

Let’s then include the plugins we need to integrate GitHub: “GitHub Pull Request Builder” and “GitHub plugin”. The first is especially interesting as it makes it possible to have Jenkins “build/pass” statuses directly in each PR (similar to Travis).

Access Data -> Global Access Data -> Add Access Data

Create access data for the repository you want to access. Provide username and password or keyfiles.

Manage Jenkins -> Configure System

  • Add your repositories under “Git”
  • Set “Manually manage hook URLs” under “GitHub Web Hook”
  • Generate token (https://github.com/settings/applications) with Github user which has access to the private repository and put the token under “GitHub pull requests builder”

Add Job

  1. Select “Free Style” project and provide a jenkins working space name
  2. Put your repository url (e.g. https://github.com/Name/repo) to “GitHub-Project” input field
  3. Select Git under “Source-Code-Management” Put your repository url again and select the access data from step 5 of this tutorial. Click on advanced settings and set “+refs/pull/:refs/remotes/origin/pr/” for Refspec and “origin” for name. Set also the branch specifier to “${sha1}”
  4. Select “Build when a change is pushed to GitHub” and “GitHub pull requests builder” under “Build trigger” Add persons or organizations to “GitHub pull requests builder” which are whitelisted for trigger an PR build request
  5. Add shell execution script for your test cases under build proceed (see below)
  6. Add “Set build status on GitHub commit” as post build action.

My shell execution script for the CakePHP apps looks like this:

cd /var/lib/jenkins/jobs/projectname/workspace/app
cp /var/www/projectname/app/Config/database.php Config/
mkdir -p tmp && chmod -R 777 tmp && rm -R tmp/*
cp /var/www/projectname/app/composer.phar ./ && php composer.phar update
Console/cake test app AllApp --stderr

It first navigates into the freshly pulled repo code and into the APP dir. Then it copies over the database.php from the current staging website (could also be some other place where you store them). It then creates tmp dirs and sets the correct permissions. The composer.phar file is also copied over and executed to pull all composer dependencies. At finally the AllApp (group) tests are executed – with –stderr in order for the session to work properly. One can then also apply -q for silent (=none) output.

Setup GitHub hooks

To trigger the build process on Jenkins, we need to configure service hook on GitHub. In order to achieve this, navigate to GitHub repository settings and configure Jenkins Hook URL for GitHub plugin. The URL format http://<jenkins-username>:<jenkins-password>@<Elastic-IP-Address>:8080/github-webhook/

Running tests

Run first test manually

Click on “Build with Parameters” and hit “Build” and you will see the first manually triggered build in progress. If all goes well the icon should be blue (pass) – not red (fail). In case you need to see what is going on, check the “Console Output” of a specific build.

Check automated tests

Commit to master and see if a build is triggered. Also open a PR and check if there is a badge displayed with the current build status.

Sugar

Badge

To have a small “build status: passing” badge in our Readme, we can install the plugin “Embeddable Build Status”. It will create a new menu entry from where you can select your badges including ready-to-use Markdown syntax if needed.

And if it is a private repo you might also want to htaccess the Jenkins web interface or protect it some other way from the public.

Code Coverage

You can install the plugin “Clover PHP Plugin”. Create a folder “coverage” in your workspace.

Add this to the above command:

--log-junit '../coverage/unitreport.xml' --coverage-html '../coverage' --coverage-clover '../coverage/coverage.xml'

And set up a “Publish Clover PHP Coverage Report” Post Build Action with the absolute path to this coverage.xml file in the workspace. In my case just coverage/coverage.xml as it can be relative to the workspace, as well. If you also want to publish the HTML coverage report (which is pretty neat), you can simply put coverage in there then.

Make sure you got Xdebug up and running (apt-get install php5-xdebug). It is a direct dependency here. Note that using code coverage reports will slow the whole process down by “x3″ at least. Usually that shouldn’t really make a difference, though.

In most cases you want to exclude few dirs. In case you are already using a phpunit.xml file which resides in your APP dir, you can easily add

<phpunit>
    <filter>
        <blacklist>
            <directory suffix=".php">../lib</directory>
            <directory suffix=".ctp">../lib</directory>
            <directory suffix=".php">*/Vendor</directory>
            <directory suffix=".php">*/tmp</directory>
            <directory suffix=".php">*/Config/Migration</directory>
            <directory suffix=".php">*/Config/Schema</directory>
            <directory suffix=".php">*/Console/Templates</directory>
            <directory suffix=".php">*/Test</directory>            
            <directory suffix=".php">*/Plugin</directory>
        </blacklist>
    </filter>
</phpunit>

You can also just create this file and also copy it over to the jenkins workspace.

Memory Limit

If you have a project of medium to large size you will soon see the memory usage for the test coverage of 50%+ way above 200MB. With more coverage and the project getter bigger, or some coverage report generated you will run out of memory with 256MB (which one would think should suffice). Raise your limit to at least 512 for the PHPUnit testing. You can use the phpunit.xml file:

<phpunit>
    <php>
        <ini name="memory_limit" value="512M" />
    </php>
    ...
</phpunit>

Cleanup

Use the plugin “Discard Old Build plugin” to keep your diskspace reasonable. Without it (especially with CodeCoverage) the disk space will soon exceed several GB. We usually keep the last 40 builds.

More

This article goes more into detail about the installation of Jenkins and also how to use additional tools like CodeSniffer, MessDetector and CO using a build file and Ant. But I didn’t check that out yet. There is also jenkins-php.org if you want to go all in :)

0 votes, 0.00 avg. rating (0% score)
 
No Comments

Posted by Mark in Testing

 

CakePHP and NamedScope for DRY conditions

15 Feb

A behavior for CakePHP 2.x

Background

I stumbled upon this fork and SimpleScope. The latter has the disadvantage of redundancy in the scope conditions when used in multiple find configs. The first was pretty much how Rails’ scopes work. But amongst other small issues it lacked the possibility of using model attributes for configuration. And both didn’t have test cases. So I decided to combine both, test the hell out of them and get the best out of the basic implementation ideas.

Basic Usage

For the behavior there is a more detailed documentation in the wiki. But here it goes.

First install/download and load the Tools plugin as documented in the cookbook or its readme file.

Attach the behavior to your AppModel:

App::uses('Model', 'Model');
 
class AppModel extends Model {
    public $actsAs = array('Tools.NamedScope');
}

Then define some scopes in your model:

App::uses('AppModel', 'Model');
 
class User extends AppModel {
    public $scopes = array(
        'active' => array('User.active' => 1),
        'admin' => array('User.role LIKE' => '%admin%'),
    );
}

Then you can use those scopes in any of your find queries:

$activeUsers = $this->User->find('all', array('scope' => array('active')));
$activeAdmins = $this->User->find('all', array('scope' => array('active', 'admin')));
$activeAdminList = $this->User->find('list', array('scope' => array('active', 'admin')));

Advanced Usage

If you also want to use scopedFind(), you will also get rid of all the many find wrappers around those scopes that will often be placed inside the models.

An example:

public function getActiveAdmins() {
    $this->virtualFields['fullname'] = "CONCAT(User.firstname, ' ', User.lastname)";
    $options = array(
        'fields' => array('User.id', 'User.fullname'),
        'conditions' => array('User.role LIKE' => '%admin%'),
        'order' => array('User.fullname'),
    );
    return $this->find('all', $options);
}

Now there will maybe also be a getActiveUsers() method and maybe a few dozen more, which all contain the same condition – which is not really DRY and might be quite error-prone if the conditions have to be adjusted (easy to miss one of the many occurrences in and out of the model).

So what would be a smarter way to approach this? Let’s try to use the above scopes here – and also use the single wrapper method. Besides the above scopes, you also need to define some scopedFinds in your model:

App::uses('AppModel', 'Model');
 
class User extends AppModel {
    public $scopes = array(
        'activeAdmins' => array(
            'name' => 'Active admin users',
            'find' => array(
                'type' => 'all',
                'virtualFields' => array(
                    'fullname' => "CONCAT(User.firstname, ' ', User.lastname)"
                ),
                'options' => array(
                    'fields' => array('User.id', 'User.fullname'),
                    'scope' => array('active', 'admin'),
                    'order' => array('User.fullname'),
                ),
            ),
        ),
        'activeUsers' => array(
             ...
        )
    );
}

The scope itself will both contain active, and the config about this scope key will be stored in a single place. So if you have some very complex condition around published (> a && < b && != c && …) this will take the overhead from multiple definitions and reduce it to a single location.

Let’s execute it:

$activeAdmins = $this->User->scopedFind('activeAdmins');

In case we need to only get a list or the count, we can adjust the scopedFind:

$activeAdminList = $this->User->scopedFind('activeAdmins', array('type' => 'list'));
$activeAdminCount = $this->User->scopedFind('activeAdmins', array('type' => 'count'));

We can also overwrite the default options:

$config = array(
    'options' => array(
        'limit' => 2, 
        'order' => array('User.created' => 'DESC'))
);
$twoNewestActiveAdmins = $this->User->scopedFind('activeAdmins', $config);

You can also get a list of available scoped finds:

$scopedFinds = $this->User->scopedFinds();

Scoped finds:

  • require a name string
  • optionally use a find array

The find arrays:

  • optionally use a type string (defaults to all)
  • optionally use an options array
  • optionally use virtualFields

The options arrays:

  • can use the behaviors’ scope property
  • support all other find options (including contain, order, group, limit, …)

Tip: See the test cases for more complex examples.

Testing

You should test your scopes, even if it’s just something like this:

public function testScopes() {
    $scopes = $this->User->scopes;
    // Each on its own
    foreach ($scopes as $scope) {
        $this->User->find('first', array('scope' => $scope));
    }
    // All together
    $this->User->find('first', array('scope' => $scopes));
}

In case there was invalid SQL, missing fields, wrong contain statements, it would be noticeable right away.

If you use scopedFinds, don’t forget to also unit test them (regarding valid SQL). This can easily be forgotten now as you don’t have the find wrapper methods anymore. In case you are lazy, add this test case to any model test that uses custom scopedFinds:

public function testScopedFinds() {
    $scopedFinds = $this->User->scopedFinds();
    foreach ($scopedFinds as $key) {
        $this->User->scopedFind($key);
    }
}

This will at least execute each find and throw an error if the SQL is invalid. It is advisable to have a more thorough test case for each find key, though, that includes the assert of the return value.

Outlook

I bet when this behavior is thoroughly used, there will be quite a few adjustments necessary. But all in all this already seems to cover most of the use cases.

With Cake3 and stackable custom finders much of the functionality we need here will be part of core functionality. Which will be awesome. Never-the-less, until then this can be a solid solution to keep the scopes/conditions DRY.

2 votes, 5.00 avg. rating (97% score)
 
No Comments

Posted by Mark in CakePHP

 

AJAX and CakePHP

09 Jan

I don’t know why I didn’t post about AJAX earlier. It is probably one of the topics that most don’t really know how to approach. The docs don’t say much about it, and so many try to rely on (outdated) blog posts out there.

Why AJAX

We shouldn’t use stuff just because it sounds fun. So sometimes AJAX is abused or used where it shouldn’t. I use it mainly where the user cannot or should not reload the whole page just to get more information or an updated piece of content which will be loaded after the page completed rendering (asynchronous). This can be part of a (sub) form, a search result or detailed information on some piece of content in a popover like window.

Basic concept

I always recommend to use the proper extension when dealing with AJAX. So if we deal with basic HTML, we can simply go for /controller/action/. See the AJAX Pagination example on how to use AJAX here as sugar on top of the basic pagination (that then serves as a fallback for non-JS-browsing or search engines). In this case it is fine to serve AJAX extension-less.

In most cases (and in all where fallbacks are not necessary) it is better to use JSON as a return type, though. It allows you to deal with the returned content in a more flexible way. That also means that is is not extension-less anymore now. Thus, you should target /controller/action.json and use the RequestHandler to get all the CakePHP magic going here (setting the correct response header application/json and switching/disabling the layout/views.

This is especially important when throwing exceptions (404 = NotFoundException, 403 = NotAllowedException). All those would then automatically be a proper JSON response message right away.

So for a basic JSON response via AJAX, all you then need is:

/**
 * Ajax action to get favorites
 */
public function favorites() {
    $this->autoRender = false; // We don't render a view in this example
    $this->request->onlyAllow('ajax'); // No direct access via browser URL
 
    $data = array(
        'content' => ...,
        'error' => null,
    );
    return json_encode($data));
}

This will not use a view or layout file and simply return a json encoded string.

But to me that looks kind of wrong. Manually using json_encode() in the controller is not something you should aim for. The view classes are responsible for the actual translation into the target representation format. So such a view class should contain that call. We only make sure that this view class gets the data it needs.

That said you could use the JsonView to make it cleaner and cut down the code by a few lines:

public function favorites() {
    $this->request->onlyAllow('ajax'); // No direct access via browser URL - Note for Cake2.5: allowMethod()
 
    $data = array(
        'content' => ...,
        'error' => null,
    );
    $this->set('_serialize', 'data');
}

Note that by using _serialize here we tell the JsonView class that we don’t need a view template to be rendered here. It will skip all the rendering and just use what you set() to the view class. Make sure to check out the JsonView docs on how to use a view template if you really need it. though. It will then automatically use a subfolder “json” and its View/ControllerName/json/favorites.ctp template here.

In the view we can then use this action via JS and the URL /controller_name/favorites.json:

$.ajax({
    type: 'get',
    url: '<?php echo $this->Html->url(array('action' => 'favorites', 'ext' => 'json')); ?>',
    beforeSend: function(xhr) {
        xhr.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
    },
    success: function(response) {
        if (response.error) {
            alert(response.error);
            console.log(response.error);
        }
        if (response.content) {
            $('#target').html(response.content);
        }
    },
    error: function(e) {
        alert("An error occurred: " + e.responseText.message);
        console.log(e);
    }
});

Note that we can first check on the error string to abort modification of the DOM if there is some error (similar to validation errors or flash message errors) occuring. And if there is some content we can then easily output that.

AjaxView

Now that is all nice, but for me that is not yet it. In this particular case we don’t just pass down model data. We render the view just as we would for normal GET access. Thus I would like it to be as close to it as possible. I want to avoid the serialize call and manual json_encode part (both in controller and view layer), just as in JsonView, but with all the basic View rendering still intact. So I created the AjaxView.

It takes care of mapping the content into the above JSON structure, and automatically tries to render via “ajax” subfolder (View/ControllerName/ajax/favorites.ctp). It will also not use any layout, just the simple HTML template code in the ctp.

A typical use case:

public function favorites() {
    $this->request->onlyAllow('ajax');
    $this->viewClass = 'Tools.Ajax';
}

The result can be of the following structure:

{
    "content": [Result of our rendered favorites.ctp],
    "error": ''
}

See the test cases for it on more cool stuff – e.g. adding some more data to the response object via _serialize.

Full blown example: Chained dropdowns

This has been one of the examples around the usage of AJAX. When you have two dropdowns and select something from the first, the second dropdown should contain updated content relevant for that selection. In our example, we select countries, and upon selection you can then chose from country provinces (if applicable).

Our controller fetches the lists of countries and country provinces:

public function chained_dropdowns() {
        $countries = $this->Controller->Country->getList();
        $countryProvinces = array();
        foreach ($countries as $key => $value) {
            $countryProvinces = $this->Controller->CountryProvince->getListByCountry($key);
            break;
        }
        $this->set(compact('countries', 'countryProvinces'));
    }

The CountryProvinceHelper component (from my Data plugin) which I usually use here basically does some more, but for a simple example this suffices. It will simply pass down all countries and the country provinces for the first country (which will be auto selected right away).

The view then contains the form:

<?php echo $this->Form->create('User');?>
    <fieldset>
        <legend><?php echo __('Countries and Country Provinces');?></legend>
    <?php
        $url = $this->Html->url(array('plugin' => 'sandbox', 'controller' => 'ajax_examples', 'action' => 'country_provinces_ajax', 'ext' => 'json'));
        $empty = count($countryProvinces) > 0 ? __('pleaseSelect') : array('0' => __('noOptionAvailable'));
 
        echo $this->Form->input('country_id', array('id' => 'countries', 'rel' => $url));
        echo $this->Form->input('country_province_id', array('id' => 'provinces', 'empty' => $empty));
    ?>
    </fieldset>
 
    The province list is updated each time the country is switched. It also has a basic fallback for POST data (will auto-remember the previous selection).
    <br /><br />
 
<?php echo $this->Form->end(__('Submit'));?>
</div>

The jQuery code can either be added to the page or stored in a separate js file:

$(function() {
 
    $('#countries').change(function() {
        var selectedValue = $(this).val();
 
        var targeturl = $(this).attr('rel') + '?id=' + selectedValue;
        $.ajax({
            type: 'get',
            url: targeturl,
            beforeSend: function(xhr) {
                xhr.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
            },
            success: function(response) {
                if (response.content) {
                    $('#provinces').html(response.content);
                }
            },
            error: function(e) {
                alert("An error occurred: " + e.responseText.message);
                console.log(e);
            }
        });
    });
 
});

Then all we need is the AJAX action

public function country_provinces_ajax() {
        $this->request->onlyAllow('ajax');
        $id = $this->request->query('id');
        if (!$id) {
            throw new NotFoundException();
        }
 
        $this->viewClass = 'Tools.Ajax';
 
        $this->loadModel('Data.CountryProvince');
        $countryProvinces = $this->CountryProvince->getListByCountry($id);
 
        $this->set(compact('countryProvinces'));
    }

and the AJAX view for this action:

<?php
if (!empty($countryProvinces)) {
    echo '<option value="">' . __('pleaseSelect') . '</option>';
    foreach ($countryProvinces as $k => $v) {
        echo '<option value="' . $k . '">' . h($v) . '</option>';
    }
} else {
    echo '<option value="0">' . __('noOptionAvailable') . '</option>';
}

To see the full code in action take a look at the live example in my Sandbox or the source code of it.

Error out

If some error occurs in your action you could either throw an exception and handle that in the frontend or pass down an error code/string to the AjaxView, disabling rendering and returning only the error to the frontend (in response.error):

$this->set('error', 'No record found');

For simple stuff it’s probably easier to use the exception free approach (using HTTP code 200). The resulting JSON would be something like:

{
    "error": "No Record found",
    "content": "",
}

But in general try to stick to exception based error handling. That would return a response with an error code of 4xx or 5xx usually (depending on the type of exception) and the following JSON object:

{
    "code": 404,
    "message": "Not Found", // Note that in Cake < 2.5 this might be "name"
    "url": "...",
}

In this example it’s a NotFoundException.

Your JS code needs to handle this gracefully. There could always be some database exception or alike thrown. So better account for that scenario.

Further improvement

You could write some generic JS code or even a generic jquery plugin to handle the AJAX response. This would cut down the lines for the query code drastically and keep it DRY.

If you want to automatically make all ajax requests respond with this view class, you could use something like this in your AppController’s beforeFilter callback:

if ($this->request->is('ajax') {
    $this->viewClass = 'Tools.Ajax'
}

It would cut down on that one line your actions.

I would not recommend that, though. as the JSON view is usually a good default. In my experience you usually need that more often, e.g. when just returning data (multiple records in particular) from the model without rendering any view.

But in general you could try to leverage a component to take care of all that and on top add even more cool things like redirect handling (output it along with the rest of the JSON data instead of actually redirecting) or flash message handling. That together could make it a real useful bundle and cut down your controller code a lot. No need to add a lot of switchs and if statements, if the code is exactly the same for both AJAX and normal requests. See my AjaxComponent which is still “beta” and could be the answer to this.

Last tips

Don’t forget to include the RequestHandler component and set up Router::parseExtensions in your routes:

Router::parseExtensions();
Router::setExtensions(array('json', ...));

This will tell CakePHP that all .json URLS are automatically served as JSON (which the appropriate headers).

Remember to not cache ajax actions to avoid old content being delivered:

if ($this->request->is('ajax')) {
    $this->disableCache();
}

I actually use that statement for all my actions/content that contain dynamically build content.

If you plan on updating the DB, never use “get”, always use type: 'post' and also assert that your action only accepts POST. “get” is only appropriate if the DB is not changed by the AJAX request, so if you only use find().

Note the difference between <option value=""> and <option value="0">. Using validation and the numeric rule we assert that the latter passes validation (when there is nothing to select) whereas the first would trigger a “You need to select a province” error of some sort when trying to save.

Using var targeturl = $(this).attr('rel') + '?id=' + selectedValue; or tricks like data('url') and data-url etc keeps the URL handling and other dynamic PHP based content out of the JS files and makes it easier to store them separately and without direct dependencies. It also avoids shortcut solutions like hardcoding URLs and such.

The xhr.setRequestHeader('Content-type', 'application/x-www-form-urlencoded'); part is quite useful as some browsers might not properly work without it (at least for me it used to).

Final note

I should mention that the AjaxView is not necessarily THE way to go. It was more an idea over night to overcome those manual render and “echo json_encode” calls and to unify some of the redundant code in my app. For your app it might be a different AjaxView. Go for it. This post, besides the AJAX topic itself, can also just show how easy it is in 2.x to extend the View class to something you can leverage to make your code DRY and consistent throughout your app.

6 votes, 4.83 avg. rating (95% score)
 
8 Comments

Posted by Mark in CakePHP