RSS
 

Posts Tagged ‘CakePHP’

Developing CakePHP 3+ Plugins, it’s fun!

29 Jan

It is fun – and it should be!

The days of CakePHP 2 plugins and how difficult it was to actually develop plugins are over.
Back in the days (OK, I still have to do it once in a while), there was even an app required to test a plugin. Since you didn’t want to have a boilerplate app for each plugin, you usually worked in your actual app. So you had cross contamination from that messing up your tests and stuff. Really annoying.
The only thing I am still missing once in a while is the web test runner, as it speeds up things for some use cases (usually with browser related output).

While most of the concrete examples are about plugin development for CakePHP 3, the main ideas apply to all library code you write. And if you are a developer for other frameworks, the same principles apply, only the concrete implementation might differ. So you could skip the "real story" part.

Well, but now to the fun part I promised in the title.

CakePHP 3 plugin development – real story

My goal was simple: Developing a simple plugin for hashids support in CakePHP in a few hours.

The CakePHP plugin docs mentioned a few basics, but in the following paragraphs I want
to go into more concrete details.

How do you start?
I started by creating a fresh GitHub repo "cakephp-hashid" and cloning it (git clone URL).

Then I added the boilerplate stuff like composer.json and gitignore file. You can either copy and paste from existing ones,
or even bake your plugin code (cake bake plugin Name) and move it to your stand-alone plugin.
Keeping in the app is also possible, but I prefer to keep it outside and develop it test driven until it is in a working state.
This way you are usually faster. TDD – test driven development – actually helps to speed up development, and you get tests with it for free.

Now it was time to set up the behavior code and the constructor setup as well as a test file.
With php phpunit.phar I already got immediate results of the initial work, and could fix those with almost zero overhead.
As soon as I added more use cases, especially with some config options and edge cases, I quickly saw where things were not working as expected.
But while getting those to run, I also saw if I broke the existing already working tests. Big help.

Once I got everything in a beta usable state, I pushed and published the repo.
After some documentation (!) in the README as well as Travis/Packagist enabling, I already composer required it in my Sandbox project and started to use it "for real".
When I got it all working in live mode, I felt convinced enough to release a 0.1 beta tag.

The cool thing that coveralls (or codecov.io etc) told me then:
I got 95% code coverage out of the box without writing additional tests (I did write them parallel to the code as "pseudo app test frame").
Simply because when I wrote the plugin code and tests, I already tried the different options available, running into all possible code lines by design.

So now I added more features over the next 1-2 hours, released 2 more tags and after a few days it is now time for the official 1.0.0 release.

Here is my sandbox demo code for this plugin, by the way: sandbox3.dereuromark.de/sandbox/hashids.

In parallel I started even a 2nd small plugin Markup, which took my only half the time even because now I was already quite fast in setting up the boilerplate stuff. Here I also tried to keep it extensible for use cases of future plugin users.

So overall I invested a few hours total to have two easily maintainable plugins that are open for extension, but by default suit my needs. Try it yourself, you will see that this way it really is not too hard to develop and publish such a plugin.

(Plugin) coding tips

If you develop a plugin for the first time, take a look at the existing ones listed in the awesome-cakephp list.
They might give you some insight in how things can look like. How we add a bootstrap for testing, how a Travis file looks like etc.

For test cases it also never hurts to take a look into the core test cases.

Plugin vs. core feature

This issue comes up every week basically. For me, beginning with CakePHP it was difficult to tell what should be covered by the core and what should stay as community plugin. I had this idea that every use case must be supported by the framework itself. Over time, it become more and more clear to me that a framework itself should stay lean and focus on the majority of the use cases and maybe provide a way to make it extensible for those edge case.

As a logical conclusion some of the CakePHP core functionality has been split off into it’s own repositories, like Acl, Migrations, Bake, Localized.
Not all of the users need those additional tools, in fact almost no one used Acl, and you only need Bake for development.

The positive side-effect here is that the iterations of these plugins can be independent from the core development, making feature development faster as there is no core release for the whole package necessary anymore.

Ask yourself the following questions:

  • Does the majority of devs need this functionality, and if so, this specific implementation of it?
  • Is it beneficial for both core and plugin to keep it standalone for maintenance and extendability reasons?

There will be more questions you can ask if you continue reading the next chapters about more generic principles.

Try to follow coding and package principles.

With CakePHP 3 we can finally adhere more correctly to some very basic coding principles. Most of you might know (or at least heard) about SOLID and Package Principles.
They following tips go into more detail what it means for our CakePHP plugins.

Coding principles (SOLID)

SOLID principles

The 5 well known SOLID principles can help us deciding how to best code our plugins and libraries.

Single responsibility principle (S)

Try to keep your classes simple and short, doing one specific task.
Take a look into my Geocoder behavior.
It does not actually geocode, because that is the task of a specific class. To avoid it doing too much, the behavior only wraps this Geocoder class and forwards calls to it. This way the only responsibility of this behavior is to help the model layer (usually a Table class) to geocode the entity data, while the single responsibility of the Geocoder class is to perform this geocoding task by talking to an API.
The additional advantage is that we can also use the library class as standalone, so we might want to provide a GeocodeShell, for which we most certainly don’t want to use behavior to encode a simple input string.

Open/closed principle (O)

Your code should be open for extension, but closed for modification. You will most likely never be able to guess and support all use cases for your code out of the box. Often times people will have come up with new ways to use your plugin.
So at the one side you do not want to have to change your code for every possible scenario. But if it was possible to support a lot of extensions out of the box, why not doing this?

If we are using dependencies in our classes, we do not want to rely on a specific class dependency, but an interface.
This might be not so important to you as plugin designer, but as soon as people use it, they might want to change maybe how a specific implementation of your dependency works.

Lets take a look at the [Slug plugin][(https://github.com/UseMuffin/Slug). It shows how you can simply set a default for 'slugger' => 'Muffin\Slug\Slugger\CakeSlugger', but it would allow any other slugger implementing the SluggerInterface. Instead of providing and versioning all possible sluggers (and their potential dependencies) itself, it allows another package to contain a user-specific slugger
to use.

So always try to not contain specific switch statements or if/else blocks. Instead, always implement against a generic interface that allows for exchangeability. This is especially important for type-hinting constructors and methods.

Here, the implementing classes just need to provide a slug() method and it will work just fine even with some BinarySlugger πŸ™‚

Liskov substitution principle (L)

Every subclass or derived class should be substitutable for their base/parent class. So make sure you make don’t widen the input/constructors, but keep them the same or narrow them.
You can always become more strict, but not less. Interfaces also help with that, as they make sure that at least those common methods have been provided.

Interface segregation principle (I)

If you create one interface containing too many different method stubs, often times you limit the possibilities of implementation.
Often times those classes can be grouped by API or Non-API, and in either of those cases need only a subset of the interface methods.
In this case it will most likely make sense to have two specific interfaces for each use case, this allows the sub-parts of your code to only rely on those relevant methods they care about.

Dependency inversion principle (D)

Ideally, we always enforce class dependencies via constructor argument commonly known as "Constructor Dependency Injection". When allowing to exchange the used class, we should respect that.

So in our case, we allow a closure to be passed into our plugin library class config:

$geocoderClass = $this->config('provider');
if (is_callable($geocoderClass)) {
	// Use the injected closure
	$this->geocoder = $geocoderClass();
	return;
}
// Manually create class
$this->geocoder = new $geocoderClass();

You never know if the class your plugin users want to use require some constructor dependencies on their own.
The DI principle should be possible for them to use, too, to fully respect the Open/Close Principle from above.

They can now have their own HTTP adapter passed into the Provider class:

$config = [
	'provider' => function () {
		return new \Geocoder\Provider\FreeGeoIp(new \Ivory\HttpAdapter\CakeHttpAdapter());
	}
];
$this->Geocoder = new Geocoder($config);

As with the first principle, it is also important for DI that if you type-hint methods, class properties and alike, always try to use the Interface, not a concrete class.

Note that there is a second way to pass class dependencies usually referred to as "Setter Dependency Injection".
This should be avoided for required classes and only be used for optional parts, if any.

Package principles

Some of the 6 commonly known package principles can also be quite useful for our decisions.

Reuse-release equivalence principle – consider the scope

It is very much possible that whatever plugin you are going to write, the functionality itself could be interesting to other PHP applications.
With 2.x it was hard to provide it both ways, with 3.x this is now as easy as it gets.

A good example is Mark Story’s AssetCompress plugin.
Back in 2.x this was a full blown CakePHP-only dependency.
He realized that most of the code behind it could very well be useful to other PHP projects, though.
With CakePHP 3 it was finally possible to move the code into a standalone lib called MiniAsset.
All that was left of the plugin code was to function as a CakePHP bridge. This enables the actual library to be used by everyone whereas the framework users could use the plugin.
r
So also check your existing (plugin) code for this maybe.

Common-reuse principle (CRP)

Code that is used together, should be ideally in the same package. Micro-splitting each class into it’s own plugin might not always be wise. It also increases maintenance time/costs.

So if you have a Geocoder Library class talking to the API and a Geocoder Behavior using this Library to populate entities, you would always use them together, or just the library class.
Splitting them into own plugins does not sound feasible here, as the default case sees them used together.

Common-closure principle (CCP)

A package should not have more than one reason to change. So when you put too many different classes or even multiple different dependencies into a plugin/lib, you might have to major bump for each of those separately, creating a lot of annoyance for the users only using the unrelated part of code.
Bear that in mind when packaging. Packaging a lot together eases maintenance time/costs, however. So you need to find a good balance here (also regarding CRP principle) for the maintainer(s) as well as the package users.

Most probably know my Tools plugin, which was started way back in 2006 with SVN and no github/composer.
At that time it would have been super painful to support multiple plugins via SVN externals and no release/version locking really.
Thus one monolith class collection. Over the time, with GitHub and more people using it (not just me anymore), it became clear this approach is outdated and troublesome. Most of the time people use only a small subset and have to get all irrelevant class changes on top.
So with CakePHP 3 already becoming alpha/beta I started to split off some of the chunks that really deserve their own plugin namespace.
TinyAuth, Geo, Ajax, Rss and Shim have been created.
Now everyone using TinyAuth or Rss, for example, will most likely use all those classes together, while inside Tools it would have been 1% of all the rest.

I will further split out stuff in the future, wherever this makes sense regarding to these two first packaging principles. But it would also be impractical to have to maintain 150 new plugins for each small stand-alone class used somewhere at some point. So a good balance is the key again.

Package coupling (ADP, SDP, SAP)

  • Prevent cycling dependencies
  • Change is easiest when a package has not many dependencies (stable), so make sure if it does those are also not blocked for change.
  • Stable packages ideally have a lot of abstraction (interface, …) exposed to the depending parts so their stability does not prevent them from being extended.

Note that the term "stability"/"instability" isn’t evaluative. It simply refers to the dependencies of a package here.

A quote about stability of classes:

Typically, but not always, abstract classes and interfaces are not dependent on others and tend to be more stable as against concrete or implementation classes. This is because the abstract classes or interfaces typically represent the high level design and not the implementation, e.g, an interface called Logger will be more stable than classes FileLogger or DBLogger.

ifacethoughts.net/2006/04/15/stable-dependencies-principle

So in both cases (class vs package level) stability means something slightly different but in the end sums up to "less reasons to change".
By trying to create and use packages and classes in the direction of stability usually is the best approach and will keep necessary changes at the endpoint (usually your app) at a minimum, as well as for most parts (plugins) in between.

Releasing

Don’t forget to release your code with tags following "semver".
You can start with 0.x and once you feel comfortable that it is fairly stable, release a 1.0.0.
New functionality usually goes into minor releases, bugfixes in patch-releases. A BC break warrants a major version jump.

Framework Semantic Versioning

With releasing plugins for a CakePHP version strict semver can be somewhat confusing, though (1.x/2.x here is for 3.x there, 3.x+ is for 4.x there, etc).
One more severe problem with that is that once you released a new 3.x framework compatible version you cannot major bump your 2.x code, as there is no number left in between. You have to break semver, or do some other workaround using composer-constraints. Some might even suggest to use a new repository for the 3.x compatible code etc.
Most of the time people are just afraid of major bumps and often use a minor one to introduce larger breaking changes.
It seems like here the design itself does not fit to the use case and misleads the maintainer to do bad things (talking from experience).
I think there is a better approach.

Something that could be considered "framework-semver" is the following approach I have seen occasionally so far:

x.y.z

  • x: CakePHP major version
  • y: Plugin major version
  • z: Plugin minor/patch version

Examples:

  • 2.3.4: CakePHP 2.x, Plugin 3.x for this major, Minor/Patch v5
  • 3.1.0: CakePHP 3.x, Plugin 1.x for this major, First Minor

So the first number of both core and plugin matches.

In my book this makes it way more clear as the plugin itself cannot live without the CakePHP core dependency and at the same time has to be compliant to each of those different major versions.
So in all this makes more z bumps, and occasionally an y bump. But that is OK.
It also requires you to look the y version then "vendor-name/plugin-name": "1.2.*", as every y bump could break BC.

In theory you could also use a 4th digit, the actual patch version: x.y.z.patch, e.g. 2.3.4.1. This would make it fully semver in itself again, but is usually not needed, as new functionality that is BC and bug fixes are both supposed to be z compliant (otherwise it would be major plugin version bump).
Either way it solves the issue of framework dependent plugins by design, and not by workarounds.

A smilar approach was already published here, but this is not composer friendly, and since all non major bumps should be BC, there should be no immediate need for a prefixing that includes the minor version of a framework.

Note: This is just some grounds for discussion so far. If you plan on using this you should really make it clear and documented it in bold visible in the README or something.
Because by default people will most likely assume you are following default "semver". Are there any downsides so far? I would really like to have some feedback here from the whole community.

License

Most people tend to forget that they release packages that are to be used in other peoples’ (proprietary) software. That means that they cannot legally use your code unless you specifically put a license in they know they can use. An MIT license is recommended and always works.
But that requires the file to be present in the root folder and/or in the files itself.
Just having a statement in the README usually does not suffice. Just something to look out for.
It would be sad if lots of projects/companies would like to use your plugin but cannot because of trivial license issues. Get it out of the way. You will benefit from more uses because of more feedback/contributions of them, as well.

Maintaining

One thing you should consider is regular maintenance for your plugins. Make sure CI (e.g. Travis) tests pass, that issues and PRs are taken care of in a reasonable time frame etc.
If that is too much to do, you can always ask for help and co-contributors, or even hand off the repo to someone else entirely.

Ready, set, …

Then take an idea and pluginize it.

You really want to start coding your own awesome plugin now but don’t have an idea? Look into the wiki of that very same repo from above. There you can find a lot of 2.x plugins that have not been upgraded yet. You could take care for any that interest you.
Or you go through your existing code and check those classes (helpers, behaviors, libs, …) for re-usability.
Also don’t forget to add your upgraded or new plugin to that list πŸ™‚

Final notes

Some people might know DIP also as IOC (Inversion of Control).
Maybe also take a look at containerless-dependency-injection-for-services, a new article of @markstory about how most frameworks make their users potentially abuse DIC and
how better code your service classes.

Book Tips / Source Materials / Interesting Videos:

 
1 Comment

Posted in CakePHP

 

CakePHP Tips 2015

13 Sep

Most tips are for 2.x and 3.x as they have been applied to both versions.

Complex radio inputs

A while back I posted a tip about deep array options for select input form fields.
With my recent path for 2.6 and 3.0 (>= 3.0.8) this is now also possible for radio() fields:

echo $this->Form->input('Model.field', [
	'type' => 'radio',
	'legend' => false,
	'div' => false,
	'options' => [
		'1' => ['name' => 'Option A', 'title' => 'A Title'],
		'2' => ['name' => 'Option B', 'data-foo' => 'bar']
	]
]);

Filtering

As of now, there is no way to directly disallow sort fields of pagination views.
There is a hack in 2.x to workaround this, though. Just set the whitelist to a non-existent field:

['foo']

This way the whitelisting is triggered and no valid field can be found. Mission accomplished.

In 3.x (>= 3.0.8), finally, we have an easy way of doing this, by passing an empty array for sortWhiteslist:

'sortWhitelist' => []

Not passing any whitelist will just not trigger the whitelisting at all.

Using the 3.x Migrations plugin for your 2.x app

I have some rather large 2.x apps that will not be migrated any time soon. But I at least want to leverage some of the 3.x stuff already where possible.
So I can include subsplits of the 3.x version via composer.
But for CLI console scripts and the migrations plugin this does not work.
I don’t want to use plain SQL upgrade files, though. And for any other migration solution I would probably also need some third party tool.
Why not going with what CakePHP 3.x seems to be perfectly bundled with? The Migrations plugin πŸ™‚

So inside my 2.x app I create a subfolder upgrade where I put a very basic composer.json:

...
"require": {
	"cakephp/cakephp": "dev-master",
	"cakephp/migrations": "dev-master",
	"cakephp/bake": "dev-master"
},
"require-dev": {
	"cakephp/debug_kit": "2.*"
},
"autoload": {
	"psr-4": {
		"App\\": "src"
	}
},
"autoload-dev": {
	"psr-4": {
		"App\\Test\\": "tests",
		"Cake\\Test\\": "./vendor/cakephp/cakephp/tests"
	}
},
...

I basically check out the 3.x version in a subfolder along with the 2 plugins I need.
I also make sure, the cli folder and the config folder are present and working.

Then I modify the app.php to automatically pull the DB credentials from my root Config/database.php.
Since I don’t want to duplicate the configs.

$file = dirname(dirname(__DIR__)) . DS . 'Config' . DS . 'database.php';
if (!file_exists($file)) {
	throw new \Exception('Cannot find database.php');
}
$content = file_get_contents($file);
preg_match('/\'database\'\s*=\>\s*\'(.*)\'/i', $content, $matches);
$database = !empty($matches[1]) ? $matches[1] : '';
preg_match('/\'login\'\s*=\>\s*\'(.*)\'/i', $content, $matches);
$username = !empty($matches[1]) ? $matches[1] : '';
preg_match('/\'password\'\s*=\>\s*\'(.*)\'/i', $content, $matches);
$password = !empty($matches[1]) ? $matches[1] : '';
preg_match('/\'host\'\s*=\>\s*\'(.*)\'/i', $content, $matches);
$host = !empty($matches[1]) ? $matches[1] : 'localhost';
return [
	...
	'Datasources' => [
		'default' => [
			'host' => $host,
			'username' => $username,
			'password' => $password,
			'database' => $database,
		],
	],
	...

I can verify if it all works by running bin/cake bake migration_snapshot Init to make a snapshot of the current DB schema in the new migration code.
This will also automatically mark this migration file as migrated.
Now I can simply add more migration files with bin/cake migrations create {Name} and then migrate the DB via bin/cake migrations migrate.

Voila!

 
No Comments

Posted in CakePHP

 

CakePHP Meetups

19 Jun

About

Meetups are a great opportunity to get in touch with other companies or people that use CakePHP.
It is a good place to share experiences and find people even that might have similar interests or who could even join your projects.

The CakeFest is only one a year, and usually not around the corner from you, meetups can be a regional thing in your city or nearby.

CakePHP Meetups Germany

We just had a new meetup this month, and it was quite a crowd, 20+ people attended.
CakePHP devs not only from Berlin attended, but due to the IPC conference nearby also some developers from further away joined in.

The talks where mainly about how companies use CakePHP, and what experiences they made so far.
My talk was mainly about how to upgrade as smoothly as possible to CakePHP 3.x.
The slides can be found here: german-cakephp-meetup-3-berlin.

Someone posted on twitter about it: dopitz/status/608683826932543488.
Good thing I didn’t wear my worn-out shirt πŸ˜›

Oh, and there is a group pic (some left already again, though): HeikkiPals/status/610475391015383041.

Findings so far

It turned out to be quite successful here in Berlin if held only once every 3-4 months. Too regularly and people miss out.
It also helped if some interesting topics/talks were provided and communicated before-hand. An extra bonus would be a prominent speaker, but
that is not always that easy to organize.

Overall, 20+ people is a good standard already and hopefully more and more devs also join us in real life in the future.

Other meetups world-wide

Check out the awesome-cakephp list regarding meetups in USA (NY), NL, FR, …

Other frameworks can do it, why shouldn’t we? πŸ™‚

Join in

If you are from Germany (doesn’t have to be Berlin) and you want to either attend an existing meetup event or create/host your own one, see meetup.com/de/CakePHP-DE/

Send your ideas for a 20-30 min talk, propose some discussion topics and be part of the CakePHP community!

 
No Comments

Posted in CakePHP

 

CakePHP 3.0 Migration Notes

06 Jun

Trying to migrate my CakeFest app for this year’s event, I made some notes regarding the upgrade process from 2.x to 3.x.
I completed them during the upgrade of my Sandbox app to 3.0. And adjusted them after RC2 and 3.0.0 stable again while upgrading 2 more apps and 5+ plugins.

Initial steps and requirements

The following were given in my case:

  • The app was already composered (as it should be for all CakePHP 2 apps) and thus "composer" was already available
  • I used a 2.x setup with ROOT dir as APP dir directly (which seemed to cause some trouble later on)
  • I used GIT for version control and supervise each upgrade step (which also caused some trouble)

Also make sure you covered the following reading material:

As a side note: It is also wise to follow the 3.0 development, read the PRs and proposed as well as recent changes.

Let’s go

By the time I wrote this I still had to tweak and fix the Upgrade tool along with it, so bear with me if something is not perfectly in sync here.

I first made sure I got a clean 2.x app with the latest (right now 2.6) changes, as outlined in previous posts regarding "how to prepare 2.x apps for 3.x".
I also used a "AppUsesCorrection" tool I have written to move all inline App::uses() statements to the top of the file. This was a left over from earlier days and will
screw up the files if not taken care off.

I also started to use the Shim plugin a while back for all my 2.x apps to make sure I already shimmed as much as possible towards 3.x, so the now required changes are minimal. If you already know you want to upgrade sooner or later, save time and shim "ahead".

Then I basically downloaded and initialized the Upgrade tool and ran the all command on my app and plugins separately.
In case something goes boom, better run the commands individually and verify the changes after each command (and commit them away).

Afterwards I adjusted my composer.json file for 3.0 and used composer update to get all new dependencies.
Here you should also make sure all dependencies like plugins are available as 3.0 versions, otherwise temporally remove/exclude them for now if possible.
Also don’t forget the new autoload and autoload-dev parts in the composer file.

A first try run revealed that I had still a lot of manual work to do in /config first:

  • config/app(_local).php
  • core.php (deprecated) => bootstrap.php
  • database.php (deprecated) ) => app(_local).php
  • routes.php

A tip here: IMO it is wise to not directly modify app.php as changes along the line will be harder to spot.
Instead I keep the file as it is (default) and just use a second one app_custom.php on top to overwrite/complete it where necessary.
I also use a third app_local.php, which is not under version control (.gitignore), to store non-commitable stuff like keys, salts, passwords etc.

Configure::load('app', 'default', false);
...
Configure::load('app_custom', 'default');
Configure::load('app_local', 'default'); // Not under version control

I found a lot of namespaces to be missing, as a lot of App::uses() have been left out in 2.x. It still worked there, as without namespaces it only needs it once per dispatching. But now it fails hard. So if you didn’t add the missing ones back in 2.x, you need to do that now at least.
I developed a tool to do that, the opposite of the unused_use fixer pretty much. This is very complex though. as it is not
always clear what package the use statements need to come from. It needs some config overhead.

I also had to remove the table prefixes as they are not supported in 3.x using my new CakePHP 3 Setup plugin DbMaintenance shell command cake Setup.DbMaintenance table_prefixes. It removed them in a few seconds. Afterwards my Table classes were able to find the tables again.

Afterwards I already tried to access a public page. Got quite a few things I had to manually take care of now:

Manual changes

Change public $uses = array('User'); to public $modelClass = 'User';. If there are multiple statements, this has to be resolved afterwards on top, using
$this->loadModel() etc.

Routes

The routes file will most likely also have to be adjusted by hand. The (admin) prefixes are the change that sticks out most.
But it is more repetitive than difficult to adjust the routes.
Bear in mind that you can easily set the fallback route class to InflectedRoute here first to handle them just as 2.x did:

Router::defaultRouteClass('InflectedRoute'); // Use DashedRoute for new 3.x projects

URLs

All the URLs usually are now more case sensitive (and CamelCased/camelBacked)

// Before
Router::url(['admin' => true, 'plugin' => 'my_plugin', 
	'controller' => 'my_controller', 'action' => 'my_action'])
// After
Router::url(['prefix' => 'admin', 'plugin' => 'MyPlugin', 
	'controller' => 'MyController', 'action' => 'myAction'])

Also make sure, you dont use the prefix values directly (admin, …) anymore, but the prefix key itself:

// Before
'loginAction' => ['admin' => false, 'plugin' => false, 
	'controller' => 'account', 'action' => 'login'],
// After
'loginAction' => ['prefix' => false, 'plugin' => false, 
	'controller' => 'Account', 'action' => 'login'],

E.g. for the AuthComponent config here. Otherwise it will redirect you to the prefixed URL instead as admin is not recognized anymore.

Auth

The auth code in the AppController and login action needed to be adjusted.
In the controller, it is not via properties anymore, but Auth->config(). The login action needs identify() and setUser() now.

Array => Entity

With the array to entity changes a lot of view files cannot be fixed with the Upgrade shell, and stuff like echo $profile['User']['id'] has to be refactored into echo $profile->user['id'], for example.
As $user['User']['id'] would be $user['id'] now, there are changes across all MVC layers to be applied in order for the functionality to work again as expected.

Custom

For all my own custom replacements I collected them and made a Custom task over time to avoid having to do this all over again across multiple apps or plugins.
I therefore forked the Upgrade plugin.

When working with date(time) fields I also had to do some special refactoring, as some older apps had 0000-00-00 00:00:00 stored as null/default value.
This is quite unfortunate, as with Carbone and Time class, this would create negative values, which blows everywhere.
So I created a Setup.DbMaintenance dates command in the Setup plugin to refactor those fields and their content into the proper value.

The same goes for foreign keys and '0' stored in wrong DEFAULT NOT NULL columns. With the Setup.DbMaintenance foreign_keys command you can also clean those up (DEFAULT NULL + NULL value).

Validation

It would be quite the task to rewrite the whole validation with all the models and their $validate properties. So here I just used the Shim plugin from above and kept the old syntax to save time. The same for relations and a lot of other model properties. It then only needed minimal adjustments, like adding 'provider' => 'table' for isUnique rule or changing notEmpty to notBlank.

Virtual fields

Mixing them with the fields array itself is not so easy anymore.
You can use closures to help out:

// For your find('all', $options) $options
'fields' => function ($query) {
	return [
		'jobtype', // Normal field
		'num' => $query->func()->count('*'), // COUNT(*)
		'fetchdelay' => $query->func()->avg('UNIX_TIMESTAMP(fetched) - IF(notbefore is NULL, UNIX_TIMESTAMP(created), UNIX_TIMESTAMP(notbefore))'), // Something more complex
		'age' => $query->newExpr()->add('IFNULL(TIMESTAMPDIFF(SECOND, NOW(), notbefore), 0)'), // Custom expression
	];
},

Locales

The Locale folder is inside src, but the subfolders changed quite a bit. It is now flat, just two-letter country codes, for Germany the po file would be located in /src/Locale/de/ now (instead of .../Locale/deu/LC_MESSAGES/).

View ctps

These template files also have to change quite a bit.
For starters, the above array to entity conversion introduces a lot of change.
Also, all static calls now have to be handled by either importing the classes via use ...; statement at the top of each file, or you can wrap them with a helper.
A quickfix would be to just class_alias() them, e.g. the Configure::...() calls would need a lot of use statements you can omit if you put the following in your bootstrap.phpfile:

class_alias('Cake\Core\Configure', 'Configure');

Now, all Configure::read() calls work again in the ctps.

Assets

If you don’t directly output your inline assets, but add them to the "scripts" block to be outputted together in the layout, you will have to change the method calls.
It used to be 'inline' => true/false, now it is:

$this->Html->css('jquery/galleriffic', ['block' => true]);
$this->Html->script('jquery/jquery.galleriffic', ['block' => true]);

In your layout ctp you can then keep the echo $this->fetch('css'); and echo $this->fetch('script'); part as it was in 2.x.

Tricky ones

Tricky as in "not ease to spot"…

The !empty PHP bug I mentioned a while back.
I had a pagination index view where I iterate over all users and display something else if there are none (yet). This fails, now, though, as the empty check will always return false:

<?php foreach ($users as $user) {} ?>
<?php if (empty($users)) {} ?>

The empty check needs to be this way in order to work as expected:

<?php if (!count($users)) {] ?>

Or, when you know it is a query finder object:

<?php if ($users->count()) {] ?>

Same with:

while ($records = $this->_table->find('all', $params)) {}

This will run endless now. Here either add ->toArray() or use a streamable result.

UPDATE Since recently (3.0.4?) you can also use ->isEmpty() as check on any Query or Collection object:

$result = $this->TableName->find()->...;
if ($result->isEmpty()) {}

find()

I used the Shim plugin and the support for find(first) and find(count), but even then you need to make sure that for find(first) you don’t forget to adjust all those $options regarding keyField and valueField which are now required to be set if you plan on using non displayField values, as the "fields" list is ignored for it (used to work to filter on 2 fields and it automatically used those).

Magic/Dynamic finders

Careful with those, like findByUsername(). In 2.x. those returned find(first) results (limit 1 so to speak), in 3.x. those need an additional ->first() appended to have the same result.

Trait or Behavior?

You might run into this when refactoring your models and behaviors.
In 2.x behaviors had the problem that they didn’t work for non-primary models, and as such where often too limited and one probably tried to workaround it using traits.
In 3.x that limitation is gone.

I think the main idea behind behaviors keeps the same: If you want to dynamically attach and detach functionality to your models, this is the way to go. Traits are too static for this. Traits, on the other hand will be necessary if you want to cleanly overwrite Table methods, see the SoftDelete trait for an example. In that case you just can’t do this dynamically.

Additionally, behaviors can more easily be configured using built-in config() and they can be aliased easily. The downsite might be speed, which is neglectable, though.
So try behaviors first, then fallback to traits IMO.

Summary

All in all quite a lot of migration steps can be (partially) automated, which will help a lot for larger applications where it would just be super-tedious to do that manually on such a scale. But most of the ORM changes need manual code changes, which makes it really a time-intensive task for medium apps and above.
Using shims, coding wisely ahead of time, avoiding hacks or non-wrapper low-level functions, all those can help to ease migration. In the end you just have to swallow the bitter pill and get it over with. It is worth it!

 
2 Comments

Posted in CakePHP

 

ACL – Access Control Lists – revised

06 Jan

With a focus on CakePHP application development.

Authentication vs Authorization

Those two are often confused. So here a little preface:

Authentication always comes first. It is about whether you are logged in or not. At this point it is not yet relevant who has access to what.
It is merely checked if the specific action/URL can be publicly accessed or needs authentication first.

After the authentication process, usually completed through some kind of login, the authorization decides on who can do what.
Here either roles or specific users are checked upon and the access is then either granted or denied.

Role Based Access Control (RBAC)

The easiest form of authorization is usually "role based access control", where you assign each other a role, and all that needs to be done is comparing
the user’s role (or roles in multi-role setups) to the required role for a specific action.
This can be realized with a single "roles" table and "role_id" in "users" table (or for multi-roles a "roles" and "user_roles" table) along with some table or file to store the
role/action relations.

Row Level Permissions

ACL is part of Authorization, the second step.
It is called "row level" access control, as it can go beyond just roles/actions combinations. Depending on the implementation it can also allow or deny specific users access to certain actions or records and make inheritable tree structures possible. So if you deny a certain user access to a node in the ACL tree(s), the sub-nodes are also prohibited by inheritance.
In its complex form (CRUD) it has even a flag for each type of action.
Such ACL trees can be quite complex to handle manually, and as such the tools provide you usually with some methods and CLI commands to make things easier.
So for the CakePHP core ACL there is even a plugin around some extra utilities for it.

ACO vs ARO

Read in the docs how ACL works regarding ACO (Access Control Object) / ARO (Access Request Object).
Essentially, ACLs are used to decide when an ARO can have access to an ACO.
While this post mainly focuses on controlling access to actions and records, ACOs could be anything you want to control, from a controller action, to a web service, to a line in your grandma’s online diary. So it allows you to make anything "accessible".

Why ACL?

First of all you have to assess what kind of authorization is needed for your application. In 95% of all cases, a full stack row level ACL is not even needed, and simple roles per user would suffice.

The main questions boil down to:
a) Does it need row level permissions as in "I need to allow/deny certain actions – or even specific records – to certain users"? Sure this can’t this be done with more roles?
If so, then go for some ACL implementation. If not (and this often times cuts down the complexity quite a bit) use a role based one.

b) Does it have to be dynamic? For row level this usually has to be some DB driven solution. For roles it is sure nice to have some backend, but in my experience
it is often not really necessary, at least if it is not some kind of CMS, but a web application that has developers close by.

So it results in 3 different outcomes:

  • Row level + dynamic (=ACL)
  • Role based + dynamic (=RBAC dyn)
  • Role based + static (=RBAC stat)

Note that the 2.x core ACL also provides an INI based approach, but that is not recommended and somewhat useless.

Problems with ACL

When working with ACL back in 1.x and 2.0/2.1/2.2 one large problem with it was speed.
It is both slow in generating the aco/aro tree from all actions available throughout the project as well as checking them.
Depending on the implementation this can sometimes be the bottleneck of some controllers and their actions.
Especially with a larger user base and a not so small app this can probably get out of hand very soon.

When building the aro/aco trees the resulting tree structure table always seemed quite fragile to me. At least a few times it got, for some reason,
broken, and repair tools had to recover the tree structure. It is also very difficult to debug due to its complexity.
Additionally, you have to be very carefully with handling the data: You cannot just delete a user manually, or reset/truncate a table. It always has to go through the ACL channels for the tree to stay valid and filled with the right data.

Another problem I encountered when working with ACL:
It is locked to the DB, instead of the code base. So reproducing some issues with accessing actions locally can become tedious as you have a totally different user-base, controlled records and thus ACL entries. And if you deploy, but forget to update the ACL (or some action is not added for some reason) people will be locked out without indication/feedback. If you upload new actions, the permissions to it will not always be deployed along with it which feels counter-intuitive to me.
That applies for any dynamic (DB driven) permission setup, of course. So role based ones with an admin backend wouldn’t be off the hook here.
Using static and "file commited" permissions here assert that those always fit to the current code base. I found that a lot easier to maintain and debug.

Static RBACs ftw

For the last 6 years I have never needed ACL for any of my applications or have it found useful for a lot of others I have examined.
Defining some good roles was enough. And it was simple. Additionally, by keeping it static I neutralized the issue with the "not-in-sync" issues above.

I see a lot of people over-engineering that part without the (immediate) need for it.
So I would really keep it as simple as possible until the use case for more complex mechanisms arises.

ACL definitely has its use cases, and I wouldn’t dare to call it evil, but it needs to be thoroughly thought through and not applied blindly.
In case you need to use it, check here, if the ActionsAuthorize (Uses the AclComponent to check for permissions on an action level) suffices over CrudAuthorize (action -> CRUD mappings).

Alternatives to the CakePHP core ACL

Alternatives on row level

Often times you can’t get around some basic row level checking, e.g. when a user may only edit/delete his own posts.

The book has examples on how to add the currently logged in user to the data upon add action, and how to check for edit/delete if that user is the owner of that record.
This can be done using the controller’s isAuthorized() method. A little bit of hardcoding saves a lot of trouble here.

Alternatives to ACL in general

Role based

  • TinyAuth – static INI files by default but could be easily made DB driven or at least backend adjustable.

Other

  • Controller based: ControllerAuthorize Calls isAuthorized() on the active controller, and uses the return of that to authorize a user. This is often the most simple way to authorize users. You can even mix role based with row level based auth if required.

Let me know if I forgot to list one. The full list of available plugins can be found at awesome-cakephp.

Last notes

In CakePHP3 ACL has been moved into a separate plugin, as it is just too much overhead for the core (since not so many actually use it) and doesn’t quite fit the 80%-rule therefore.

 
2 Comments

Posted in CakePHP

 

Why CakePHP?

23 Dec

A Xmas post from Germany.

My last 6 years with CakePHP

I am now with CakePHP for over 6 years (wow.. time goes by!), the second half of it as core member. I started with it back in the days where 1.2 had just come out. I was just playing around back then.
Soon I started to write the first small 1.2 apps, mostly private stuff and ideas in my head. A "phonebook" app, "recipes" site, a "sandbox" for trying out Cake/PHP things, a small social network approach and few plugins on the side.

Not long after that I helped to develop the first real CakePHP app meant to be thrown out there for commercial purposes – and since then never stopped.
A little bit over a year ago I managed to finally upgrade even the last live app to the latest version 2.2/2.3 at that time.

Disclaimer: This is not meant to be a rip-off from josediazgonzalez’ why-to-actually-choose-cakephp/. It is my personal opinion why I have been sticking to this framework for the last years.

So why CakePHP in the first place?

There are alternatives, Symfony, Zend, Laravel, CI, …
But they all didn’t convince me in the beginning – and still don’t.
Mostly because as a beginner the first steps seemed and seems to be more complicated than with CakePHP. You need to do more to see actual and usable results.
But I don’t want to go into detail here too much.

A framework is an absolute must

At least as a beginner. But even as a pro you are sometimes happy to not have to modify all those "core changes" in your code when PHP, MYSQL or any other external script is used differently over time. You are also grateful for the community effort of being able to find and fix bugs as a group effort. And last but not least, you could never come up with all those great new features and ideas all on your own.
I found CakePHP to not only be community-driven but also very intuitive – over some time anyway.

CakePHP was already easy at the beginning

I already had PHP/MYSQL experience, but even without I might have succeeded with the first trial apps.
And although CakePHP has lots of conventions and opinions on certain things it still allows you in many ways to use your own way of doing it.
Since 2.x even more than ever before.
Once you understand what goes on behind the hood you start to like it even more. You are able to contribute to the framework code as well as the documentation directly.

My personal reasons to still stick with CakePHP

Well, first of all, after so many years working with it and contributing you kinda get attached to it. Within the last three years I rose to the one of most active core developers besides the master Mark Story πŸ™‚ After what one helped the framework to become it would not make too much sense to just hop on to the next framework without a good reason.
So here the hard facts:

Less coding time

CakePHP is all about reducing development time. So most of the things are already decided for you in best-practice approaches and you can overwrite only what you want to.
Using bake and other tools you have a usable app in no time. Adjustments in your code are then also just a couple of quick changes. Fast development means less costs and earlier results.

Fast core update/fix progress

Issues or enhancements usually get added pretty quick. From a few hours (happens quite often actually) to a few weeks. The less complicated the issue and the better your initial work the faster it gets approved and included. So providing a detailed report and a patch in form of a Github pull request can really speed up the process.

Plugins

There are already quite a few very useful plugins or at least plugin-able snippets by very experienced programmers. You can easily use and enhance those.
I use my own plugins like Tools, Setup, … with various useful stuff to enhance the core and my apps even further in a DRY (Don’t Repeat Yourself) way.
That’s also one of the reasons why changing would be painful: I would miss so many of my already written Plugin classes I need every hour of every day. The overhead of rewriting it would probably be quite immense.

Busted myths

CakePHP is dead/legacy

Yeah, right πŸ™‚ That’s like super-lame to say. Check out the Github changelog and you will see how dead it really is.
Fact is, that with CakePHP3 the activity goes to a new all time high throughout the timeline. And that is just the beginning. Fact is also, that until now, it was a very constant log of activity and improvement.
Compared to other (very new) frameworks out there, that is quite the achievement. Most frameworks come and go as quick as the wind, but CakePHP is solid as a rock in
its presence and how it improves. People/Businesses can actually rely on it.

A far more interesting question would be: Why isn’t Cakephp popular despite being one of the earliest PHP framework to be written?.
People can get easily deceived by sparkling new "coolkidz" stuff. Backwords compatibility and reliability has’t been as important to any other framework. So they could progress to cutting edge in a faster pace, of course. There is a certain trade-off here to make. Nonetheless progress has been made, and that quite remarkable.
See for yourself.

CakePHP is slow

Yeah, that was partly true in 1.x due to missing lazyloading. So many things have been included without actually needing them. It could consume quite a lot of memory, as well. Since 2.x and lazyloading the developer is responsible for only fetching the data needed and making sure that caching and other speed/performance enhancements have been accounted for. Then Cake is not really that much slower than any other Framework or PHP script.
Sure: CakePHP 2 is still much more full-stack than most slim PHP frameworks. It might also not be cutting edge regarding speed. But with a little tweaking of server and app this difference is almost not measurable anymore.
And the upcoming version 3.0 beats even most other frameworks, as with composer autoloader dump and some minor caching layer it runs pretty darn fast. Even the routing, which often was a bottleneck, has been sped up a huge factor.

This is not even a good argument for comparison, as with server configuration this can easily be overcome anyway.
And what is often forgotten: CakePHP is actually super-fast in development time compensating for any speed issues compared to vanilla PHP coding.
If I had a project to finish, I would rather use CakePHP and 1/4 of the development time (and costs!) and invest those in better server infrastructure (fully compensating any speed issues and going beyond). The outcome: You still pay only half the costs πŸ˜‰

The documentation is outdated and incomplete

Total nonsense if you didn’t travel back in time. The documentation improved greatly since 2008 and Cake1.x. You could say that everything is covered and some things even more than you would probably need. Even the migration guides up until 2.6 (not released yet) are 100% complete and the code changed documented in the book in detail.
So if you don’t find something it must either be a really rare use case or you are not quite capable of using Google or the book internal search. Well, that is too harsh of course. We all overlook things sometimes. Drop in the IRC #cakephp channel and ask, if you really can’t find it, and you will be helped in seconds/minutes.

For CakePHP3.0 the focus was even put on documentation: Writing thorough docs for each part that has is been added in sync with the actual feature PR.
So far the result should be quite remarkable for a not-even-yet-RC version.

CakePHP is too much convention over configuration

I sometimes hear people complain about too many conventions and that you can’t configure CakePHP the way you like/need it. This actually was true in earlier versions.
But in 2.x this also improved quite a lot. You can pretty much configure/adjust the whole framework as you need it – either via extending, aliasing, replacing core files or events/callbacks.
Bear in mind that conventions are usually a good thing. They – in a default scenario – help you to get it done very quickly. If you start to reach an edge case you can still adjust accordingly, but first try to stay within the conventions for your own benefit.

With CakePHP3.0 this even goes one step further:
You can leverage the whole conventions stack if you want to, or you can go totally down your own path. The latter I wouldn’t recommend, though, as you really save a lot of config overhead when using conventions wisely.
Also: Using namespaces, there is pretty much no limitation anymore. Classes can be finally named the same without fatal errors. A simple use x as y and it’s resolved.
You can replace complete parts of the application. Since major parts like "bake" and "translation" are moved out of the core, you can also more easily extend and adjust those now.

My learnings

Over the years, you also learn from your previous work – and mistakes.
And we all make em. That’s natural.

Documenting is important

I didn’t really think of documenting in the beginning. Over the years there were many times were I had to regret that for various reasons.
So new plugins and functionality get documented from the start, making it transparent both for me and others what to expect.
I follow the approach to put the dochs directly into the repo as subfolder, containing a README.md as overview. This is very handy (close to the code) and readable (automatic markdown parsing online).

Write tests

I also didn’t test much when I started to develop. Working with CakePHP showed me how simple and yet effective it can be to do that though.
You save yourself from a lot more regressions and it serves as part of implicit feature documentation, in case your forgot that.

Don’t reinvent the wheel

Yeah, back in 200x I had to write everything myself. But the systems around us evolve quicker and quicker. It is just impossible to keep up with everything and still write new application code. So if there are existing libs or services that do the job, I try to use them now. An example is "Carbon" for datetime or "Mailchimp/Mandrill" for sending emails/newsletters.

Keep code clean

Refactor once in a while to keep the code clean and readable. This makes understanding and extending it afterwards easier.
Follow best practice approaches where applicable. Use a documented coding standard and stick to it.

Learning from other code and frameworks

Other frameworks have different approaches, they are valid for some use cases nonetheless. I had to look into Symfony2 and alike for one or two projects and even if I wouldn’t use them for my RAD (Rapid Application Development) projects, they can be useful for other projects. I could also once in a while get some very good ideas from them on how to approach certain things, and sometimes even let them mold into CakePHP functionality.
Similar things can be said about existing library code out there. So be open minded and take a look over the "comfort zone" once in a while and suck in other people’s ideas and approaches.
And always check if the tools you use still are the right ones for the task. If at some point something could be better handled with a different framework, I would probably look into it. But knowing my "tool" quite well, I didn’t see anything yet that wouldn’t have been handled by it so far in an appropriate way for the tasks at hand.

Splitting Utility repos into smaller chunks

I wouldn’t exactly call this a mistake, it was more the logical choice back then with SVN (and later basic GIT). My Tools plugin got pretty loaded with stuff, just because it was easier to include a single plugin than trying to svn:externals or git submodules separate ones. The overhead was not acceptable.
Now in 2014+ with composer as de-facto standard, it looks a whole lot different. As such, I try to outsource some functionality of it into their own plugin repos, e.g. Geo and TinyAuth for 3.0. More should follow.

Which brings me to the last point which will also be easier with smaller repo code:

Follow some kind of semantic versioning

With composer this is an even more important task than before: SemVar is necessary to prevent incompatible libraries.
So if you develop new features for a plugin for one app, other apps using the old version should still function normally.
By always using a single branch directly, this can soon blow up. Using semantic versioning and releasing new versions (version numbers are cheap)
we can assert that everyone is happy and code breakage stays at a minimum.

What’s next

I will try to help with the release of 3.0, the next major step in PHP framework history, as much as possible besides my day job and extracurricular activities.
Earlier versions of CakePHP sure lacked the "coolness" of new PHP features, letting quite a few users drift off to other frameworks. I can understand that partially. We all want to use the newest and geeky stuff. If it is really necessary (at that point in time!) is a different story.
Anyway, CakePHP 3 will catapult itself to the very front, and probably beyond most other PHP frameworks. And rumor has it that this time it will try to stay on top – with a fairly high chance of succeeding.

So I am really looking forward to how this will change the balance of framework usage here again, and how probably many of laravel’s or symfony’s followers will jump back to the framework with the longest history of all. Stay tuned for more posts on how to prepare 2.x apps for 3.x so that the final migration will be as smooth as possible.

Don’t forget to spread the word about CakePHP(3). Blog, twitter, write plugins (or upgrade existing ones) …

Additional Resources

For German developers there’s a very recent podcast where I go into details about CakePHP.
The (English) podcast.cakephp.org one is mainly about upgrading between the different versions of the framework.

If you want to stay updated, subsribe to the blog RSS feed or follow my on twitter.

Merry Xmas everyone

And a great start into the next chapters of coding!

 

UTF8 can be tricky – especially with PHP

15 Aug

Everybody uses (or should!) UTF8 these days. An easy and fully supporting PHP version I did not come across yet, though.
It seems there is sometimes more to it. This article is supposed to guideline the basic setup of a CakePHP app using UTF8 and will go beyond that to the really tricky parts regarding the de facto standard encoding these days.

Note: this post is really long overdue and was in my draft folder for 2+ years. So here it is, quickly published before it got even more dusty^^
And dusty sure is the right word with (hopefully) no one using ANSI/ISO-8859-1 anymore these days.

UTF8 and PHP

Use the mb_ functions if you know that you real with strings than can contain UTF8 chars. So if you want to count the length of such a string:

$length = mb_strlen($string);

If you are simply manipulating strings, you do not always have to use those slower and UTF8-aware fnctions, though. But in doubt always do so.

UTF8 and preg_match()

Now this is a tricky one – especially if you don’t want to recompile PHP with the PCRE UTF-8 flag enabled or if you don’t know about it at all. IMO that should be the default, but it usually isn’t it seems.

Most times, when dealing with UTF8 strings, the /u modifier and p{L} helps:

preg_match('/^\p{L}[\p{L} _.-]+$/u', $username, $matches)

In other cases you might have to add (*UTF8) in your pattern.

UTF8 and CakePHP

CakePHP setup

The main parts are handled in the book, especially in the getting-started section.
But the main part that sometimes people get wrong is that the APP encoding is "utf-8" while in the database.php its spelled utf8.

Make sure you save all files as "UTF8 without BOM" via your IDE as soon as they start to contain UTF8 chars. Failing to do so will cause output issues.
I usually try to avoid this and use Locale translation and mainly English chars in all files as much as possible.

Note: Before adding any UTF8 chars to files, those files are always ANSI (there is no way without the BOM to distinguish those two encoding formats as they are one and the same here). So no matter how often you try to save them as UTF8, they will always still be ANSI. In case you wondered why it falls back to it again in most IDEs.

Correcting PHP functions

Some PHP functionality has been wrapped in CakePHP to overcome deficiencies regarding Unicode.
String::wordWrap() for example replaces the faulty wordwrap() function.

I also added a few fixes to my Tools plugin as Utility/Utility class:

  • pregMatch(): Unicode aware replacement for preg_match()
  • pregMatchAll(): Unicode aware replacement for preg_match_all()
  • strSplit(): Unicode aware replacement for str_split()
  • pregMatchAll(): Unicode aware replacement for preg_match_all()

Probably more to come..

Proper validation

Make sure your validation is unicode aware – that’s probably one of the most made mistakes from mainly English speaking devs/people.
They maybe assume that it works to simply use strlen() or a [a-z] regex or alike – not taking into account that for example many normal first/last names contain a lot of special chars.
Validation here should never be too strict. Otherwise a lot of people will be very upset.

So in the above example we do NOT want to use

preg_match('/^\[a-z][a-z .-]+$/i', $firstName)

but something more like

preg_match('/^\p{L}[\p{L} .-]+$/u', $firstName)

to validate a first name.
IF we actually have to validate this further than a simple "not empty" check is a different topic (I don’t think so). But if you really must, PLEASE do not shut people out because their parents gave them non-English names πŸ˜‰

A similar thing I had to fix in the core a while back, regarding domains/urls.
And this is CakePHP2.5 – the current master – so that topic sure is still quite current for some cases. More and more so with further internationalization.

Checklist for your CakePHP app

  • Ideally, use utf8_unicode_ci as collasion for your DB
  • Your layout should contain <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  • The apache/nginx should serve files as UTF8 via header Content-Type text/html; charset=UTF-8

Outview

Only in PHP7 (as PHP6 got skipped) there will be a more built-in approach then for UTF8. Until then (and maybe even then) we will have to fight quite a lot here for the next years.

There are even a few popular projects in GitHub around the UTF8 issues, e.g:

  • https://github.com/nicolas-grekas/Patchwork-UTF8
  • https://github.com/heartsentwined/php-utf8
  • https://github.com/voku/portable-utf8
  • https://github.com/gatsu/UTF8

Might be worth checking out.

Anything missing? Please let me know.

 
4 Comments

Posted in CakePHP

 

All new CakePHP Tips Summer 2014

05 Aug

For CakePHP2.x

Awesome CakePHP

Check out this all new awesome CakePHP list: awesome-cakephp
Star it, fork it, enhance it πŸ™‚

Note the cake3 branch that will proabably soon be filled rapidly with all new shiny CakePHP3 resources and plugins.

There are also a few more generic awesome lists links in there.

CakeFest 2014 coming up

Check it out – and be part of it. You can read about last year’s event here.

This year is going to be awesome (again that word^^). And not only regarding the temperatures in Madrid πŸ˜‰
A few well connected people like @philsturgeon pulled some strings and the event is listed on sites like php.net.
Let’s make this the largest fest ever.

Note: My attendance app only lists a very small subset of all people actually joining the event!

Finally a book on CakePHP2.5+

CakePHP Core Developer @josegonzalez wrote a cool book on CakePHP: Rapid Application Development with CakePHP 2.
It has some real insight in topics that are not covered by the online CakePHP documentation.

Hot tip: Use the promo code DEREUROMARK when purchasing the bundle to get 20% off your purchase!

Deployment with composer and script hooks

This is already widely used in Symfony2, for example.
Here it fires the following scripts after installing/updating to clear the cache and re-build assets etc:

"scripts" : {
	"post-install-cmd" : [
		"Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
		"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
		"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
		"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
		"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
	],

The list of available commands be found here: getcomposer.org/doc/articles/scripts.md

Why not leveraging this for your Cake2.x app, as well?
You could re-built AssetCompress files, clear the persistent/model cache, run custom post-update hooks and so on. (DB) Migration could also be hooked in this way.

Post links inside Forms

Most probably didn’t see this change. But it allows postLink() to be used inside forms now.
The issue was, that forms don’t work inside forms. And that post links contain those inline, breaking the form around it.

You can now easily create "Delete" links or other POST links inside your forms. It can write the forms to a View block which you can output later on then.
All you need to do is set 'inline' => false:

echo $this->Form->postLink('Delete', array('action' => 'delete', 1), array('inline' => false));

And output your buffered View block after closing the form:

echo $this->View->fetch('postLink');

This also works with multiple postLink() calls. The View block would simply contain all of the forms then.

Prevent internal field names

I try to stay away from internal framework used names for fields. Once you attach Search plugin or any other additional functionality on top of your
actions, this could get messy otherwise.
The Paginator, for instance, uses "limit", "sort", "direction", "page". In order for them to not conflict with search fields on existing fields, I avoid them and
use "order" instead of "sort" or "max" instead of "limit" for field names. In case this is not avoidable or a legacy DB schema, one can add the model prefix, though: "MyModelName.sort" and would also be able to resolve the conflict. But if you can already do so in hindsight it is probably smarter.

Array and ArrayObject

The new CakePHP3 core uses ArrayObject already quite a bit.
In 2.x you probably don’t use it too much yet. But it is wise to voice a warning regarding on some flaws of ArrayObject.
This article describes it.
Basically, empty() and ArrayObject don’t play nice. It only works with count():

if (count($options)) {
	// Yes
} else {
	// No / empty
}

The alternative would be to cast it prior to checking on it: (array)$options. Then empty() would work again.

The same issue you might face in index views and alike, where you want to display a subsitute text if no records can be listed in the pagination yet:

if (empty($users) { // BAD - better use count() here then.
    echo 'No users enlisted so far';
}

This would also always fail now in CakePHP 3 without any indication whatsoever as objects replaced the arrays and render empty() invalid as check here.

The rest of those issues in the article are also quite interesting. Check them out, as well.

A complete list of "flaws" has been put together on this page. It reads like the bible of Anti-PHP, tough πŸ™‚
Quite entertaining even in some parts. I would just read it as an informative page, though. It is good to know those things, but it is also easy to avoid or work-around them.

MySQL and returning integers

Some probably wondered why integers and other non-stringish values is often returned as string.
This has nothing to do with CakePHP itself, but with the PDO extension of PHP underneath.

But you can easily disable that with the following flags in your database.php config:

'flags' => array(
		PDO::ATTR_STRINGIFY_FETCHES => false,
		PDO::ATTR_EMULATE_PREPARES => false
	)

UUIDs as char36 vs binary16

UUIDs (Universally Unique Identifier) can be used as an alternative to the normal AutoIncrement IDs for primary keys. Use them wisely, they are larger and slower than than the AIIDs.
But if you use them, try to prevent char36.
According to tests such as storing-billions-uuid-fields-mysql-innodb, it is 6-8 times faster to use binary(16) instead of char(36) for UUIDs.
CakePHP 2.x+ supports binary just fine.

PHPUnit and Windows

You might have seen something as descibed in this issue: Weird output in your console when running unit tests.
This is mainly because the default setting for PHPUnit in the phpunit.xml(.dist) file is usually colors="true".
You can easily overwrite it in your own file and the weird output stops:

<phpunit
	colors="false"
	...

Or you can try to use the library linked in that ticket. I didn’t try that one yet.

 
No Comments

Posted in CakePHP

 

CakePHP 2.5 arrived + CakePHP Tips 2014

13 May

You probably read my last tip sections.
And I started to move some of them to my sandbox app.

But once in a while it might also be nice to publish a few selected tips, here as well.

Oh, and CakePHP 2.5 is out! Get it – now πŸ™‚
See the last chapter of this post about why you really should upgrade ASAP. And it doesn’t matter if you
are on 1.x or 2.x.

URLs in CLI

In CLI and your shells there is no HTTP HOST and therefore usually no absolute URLs.
But in order to properly work with links/urls (e.g. sending batch emails), we need that to work.
So I use Configure::read(‘Config.fullPageUrl’) here.
My configs.php file then contains:

$config['Config'] = array(
	'fullPageUrl' =>  http://www.myapp.de // The primary URL
)

In case you have a different domain for local development like http://myapp.local and you want that to be the fullPageUrl, make sure you overwrite the default in your configs_private.php file:

Configure::write('Config.fullPageUrl', 'http://myapp.local'); // Usually defaults to the (live) primary URL

And in the end of my bootstrap file (after including all config files), I simply do:

if (php_sapi_name() === 'cli') {
	Configure::write('App.fullBaseUrl', Configure::read('Config.fullPageUrl'));
}

To test the current domain/fullBaseUrl, you can use my Setup plugin command cake Setup.TestCli router.
It will output both relative and absolute URL examples generated by the Router class with your current settings.

So on the live server then it will output http://myapp.local instead of http://localhost when generating Router::url()s in your shells.

Careful with running shells in CLI

Most are probably not aware, but running shells in CLI needs to have a proper user management around them in most cases.
Imagine yourself running your apache as www-data (default) and log in as root or any other user not affiliated with that www-data user/role (bad idea).
Once you execute a shell and tmp cache data are (re)built, your www-data user cannot access them anymore, losing the ability to cache and triggering a lot of errors.
So make sure you only log in with a user that shares the role of www-data at least, so that both can access each others’ generated files.

A popular example is the ClearCache shell which re-builds your cache dirs in debug 0 (when changing files or db schema makes this necessary).

PS: Of course you could also switch to another cache system than the default File engine. But most probably didn’t do that yet, either.

Merging arrays

Ever wondered what Hash::merge(), array_merge and array_merge_recursive have in common – or don’t have in common?
Check out these merge comparison examples.

See what the requirements are – and use the appropriate merge method then.

There is also the + operator, which is quite useful when merging flat arrays and string key based options. This is quite commonly used in the core to merge
options and defaults:

$defaults = array(
	'escape' => true
):
$options += $defaults;

In this case the $defaults are merged on top of $options, but only if that key has not been specified yet.
This kind of merge is really efficient and fast (4-5x faster than array_merge() itself) – but should really only be used if all keys are definitely strings.

Paginating and merging

A propos merging: When setting up paginate settings in your controllers, try to prevent

public function index() {
	$this->paginate = array(...);
}

This way you kill all defaults you might have set via public $paginate or in your extending controllers (AppController’s beforeFilter() callback for example).

So it is better to use:

$this->paginate = array_merge($this->paginate, array(...));
// or
$this->paginate = array(...) + $this->paginate;

In my 2.x code for example I have this snippet in all my AppControllers to have query strings for paginations:

public function beforeFilter() {
	parent::beforeFilter();
	$this->paginate['paramType'] = 'querystring';
}

This will only work with proper merging of defaults and custom settings.
I prefer the latter because the settings are string based and here the + operator is the fastest and easiest way of doing things.
Once the key is already set in your method, the default will be ignored right away (with array_merge() and nullish values this can be different/unexpected).

And remember to not mix controller and component pagination.

Pagination and sort default order

Adjust your bake templates so that some fields like created/modified are ordered DESC instead of ASC per default in the index actions.
For those fields the first click on the header row should display them DESC right away as one would then most likely be interested in the
latest changes. Same goes for most date fields like "published" as well as fields like "priority", "rating", …

That’s how the baked code (or manually adjusted one if done afterwards) could then look like (index.ctp):

<th><?php echo $this->Paginator->sort('name');?></th>
<th><?php echo $this->Paginator->sort('amount');?></th>
<th><?php echo $this->Paginator->sort('priority', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('status', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('publish_date', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('modified', null, array('direction' => 'desc'));?></th>
<th><?php echo $this->Paginator->sort('created', null, array('direction' => 'desc'));?></th>

Using modified model data in the form again

Some of you might have had the wish of posted data that was modified in the model due to beforeValidate/beforeSave to appear modified in the view again (so
that the reason for validation errors might be more clear etc).
So let’s say you have a beforeValidate callback to clean the input of a textarea:

public function beforeValidate($options = array() {
	if (isset($this->data[$this->alias]['comment']) {
		$this->data[$this->alias]['comment'] = $this->_clean($this->data[$this->alias]['comment']);
	}
	return true;
}

So in this case it could easily be that _clean() removes some invalid content and thus the minLength rule is suddenly triggered.
Which is weird, since we posted at least twice the length of text.
To clarify to the user what is going on, one could adjust the error message – but one could additionally return the modified (ready to save) data instead of the
actually posted data.

public function add() {
	if ($this->request->is('post')) {
		$this->{$this->modelClass}->create();
		if ($this->{$this->modelClass}->save($this->request->data)) {
			$this->Session->setFlash(...);
			return $this->redirect(array('action' => 'index'));
		} else {
			// Here we assign the modified model data back to the request
			// object and therefore to the view/form
			$this->request->data = $this->{$this->modelClass}->data;
			$this->Session->setFlash(...);
		}
	} else {
		// Default values for the form
		$this->request->data[$this->modelClass]['status'] = true;
	}
}

The input field will now contain the content that was served to beforeValidate(). And combined with a good error message this will probably clear things up.

PS: You can also directly use the model’s name instead of {$this->modelClass}, e.g. ‘Comment’.

MySQL – MyISAM vs InnoDB

InnoDB is a little bit more robust as it allows transactions. Especially with CakePHP and "multiple queries" most of the time (per default) this can be quite helpful in keeping the DB in a valid state.
Also read myisam-versus-innodb on pros/cons for each.

One additional problem with InnoDB, though: Per default it creates a file that always increases in size, never decreases again. This can soon be a nightmare with backuping when it becomes >> xx GB of size.
See how-to-shrink-purge-ibdata1-file-in-mysql how to avoid that by not using that file, and instead using innodb_file_per_table.

Testing

Testing Controllers

I stumbled upon a few issues with testing controllers – especially plugin controllers.
For plugin controllers to be testable unfortunately you always need to use $uses, even if it wasn’t necessary due to conventions.
So for your Tools.TinyUrls Controller you would need

public $uses = array('Tools.TinyUrl');

Otherwise it would always try to look for the model/fixture in your app dir, which eventually always fails.

Do not forget to use --stderr when testing in CLI. Otherwise all tests that use the session (and which work fine in webtest runner) will fail:

cake test app AllController --stderr

Test Coverage

If you want to generate a HTML overview of all your locale test coverage:

cake test app AllApp --stderr --log-junit tmp/coverage/unitreport.xml --coverage-html tmp/coverage --coverage-clover tmp/coverage/coverage.xml

The report index.html will be in your /tmp/coverage folder.

More on testing – especially controller testing – can be found on the corresponding Tools plugin Wiki page.

Upgrade (deprecated) code

It is always good practice to upgrade to the current master. Not only for CakePHP, but there especially. It will save you a lot of time in the long run, as migration will be easier and faster in small steps instead of one huge step. It will also make it easier to use the new features and more importantly will also come with a lot of fixes and method corrections that the older versions didn’t get anymore. Those outdated versions usually only get security-bugs fixed. So if you look hours for an error that is already fixed in the current master, it was just a huge waste of time. I have seen that a thousands times – on IRC and elsewhere.

So in case you are not using the current master (2.5), do it now. Internally, upgrading 2.x is a "piece of cake".
Upgrading from 1.x is also not that big of a deal – just needs a little bit more manual adjustments. For most things you can use the core UpgradeShell as well as my Upgrade plugin.

In case you are already upgraded to 2.5, you can and should also remove deprecated functionality in favor of the right one.
Those deprecated things will only add to the file of upgrades for the next 2.x release or 3.x. And using the upgrade shell it is usually just one single command to execute.
So for 2.5, you should have removed all the "easy" stuff that will clearly be switched with a different way of doing things as it is mentioned in the migration guide, e.g.

  • loggedIn() in favor of Auth::user(‘id’)
  • CakeRequest::onlyAllow() in favor of CakeRequest::allowMethod()
  • Use first argument (string|array) instead of var args for CakeRequest::allowMethod(), AuthComponent::allow(), etc
  • $title_for_layout in favor of $this->fetch(‘title’) and all other deprecated view vars

From 2.4 and below there are also a few left overs that could easily be corrected:

  • FULL_BASE_URL, DEFAULT_LANGUAGE, IMAGES_URL, JS_URL, CSS_URL to Config variables
  • Remove CAKEPHP_SHELL, IMAGES, JS, CSS usage
  • Simplify HtmlHelper::css()
  • Remove Sanitize class usage in favor of a more sane approach
  • Simplify CakeRequest and PaginatorHelper usage with param() if applicable
  • Don’t use request shortcuts anymore – like $this->action in favor of $this->request->action etc
  • Get rid of Model::read() calls – in 3.x this will be get() – I use my Tools plugin MyModel::get() method here already for years
  • Use the new Model::save() syntax for options
  • Completey get rid of named params in favor query strings (Use my Tools plugin CommonComponent and Configure value App.warnAboutNamedParams to detect left-overs)
  • Replace all Set calls with Hash calls, as Set is deprecated. Make sure it is tested thoroughly (as the functionality of Hash might be slightly different).
  • Prevent using Validation::range() in favor of a custom validation rule for inclusive ranges to easier migrate to 3.x – or simply use my Tools plugin MyModel::rangeInclusive() method.
  • Further deprecations in favor of the 2.5+ way to do things

and so on.
For some details see Tips-Upgrading-to-CakePHP-2.x.

My Tools plugin also contains a few further tweaks that can help ease migration from 2.x to 3.x See the full list on the corresponding Wiki page.

This will help making sure any future upgrade is less painful.
Think about it: When you do that now the remaining TODO list will only be half the size and look a lot less intimidating. When the time comes to upgrade to 3.x it will look quite more doable.

My codez is now officially all 2.5 – and as 3.0 ready as it can get πŸ˜›

And what always helps a lot is to code clean and structured. A code mess will always be difficult to maintain. So use coding standards and enforce them. Use best practice approaches. Refactor once in a while to prevent a mess from happening.
Happy coding πŸ™‚

 
1 Comment

Posted in CakePHP

 

CakePHP flash messages 2.0

21 Apr

This is a topic often discussed. What is the best approach?
While I might not have the best, I sure have a fully working one since 2008 and CakePHP1.3. I still use it in all 2.x apps.
It is easy and does not require a lot of overhead.

Basically, it enhances the core one with the following features

  • Different types possible (now in 2.x kind of in the core, as well, using elements)
  • Multiple flash messages per type possible (with a max limit to avoid session flushing)
  • Transient messages (via Configure) and real ones (via Session)
  • Transient ones can also be fired from the views (to display some hint/info for this page) – although one should try to stick to the controller

Demo

sandbox/examples/messages

How does it work?

We attach the Common Component and the Common Helper to the AppController:

public $components = array(..., 'Tools.Common');
public $helpers = array(..., 'Tools.Common');

In our layouts we need to switch the default output to ours:

echo $this->Common->flash();

And we can use `em:

public function add() {
	...
	if ($this->Entry->save()) {
		$this->Common->flashMessage('The Entry has been saved.', 'success');
	} else {
		$this->Common->flashMessage('The Entry could not be saved. Please check the form.', 'error');
	}
}

Styling

We style our flashmessages via CSS

div.flash-messages {
	width: 90%;
}
div.flash-messages div {
	padding: 10px;
	padding-left:40px;
}
div.flash-messages div.error {
	width:100%; border-style: solid; border-width:1px; border-color:#B84D17; 
	margin-right:2px; color:#000000; margin-bottom:8px;
	background: url(/img/css/layout/icon_error.gif) left center no-repeat;
	background-color:#F7C6A5;
}
div.flash-messages div.warning {
	width:100%; border-style: solid; border-width:1px; border-color:#D0C130; 
	margin-right:2px; color:#000000; margin-bottom:8px;
	background: url(/img/css/layout/icon_warning.gif) left center no-repeat;
	background-color:#F6F3A4;
}
div.flash-messages div.success {
	width:100%; border-style: solid; border-width:1px; border-color:#009900; 
	margin-right:2px; color:#000000; margin-bottom:8px;
	background: url(/img/css/layout/icon_success.gif) left center no-repeat;
	background-color:#A5F7A8;
}
div.flash-messages div.info {
	width:100%; border-style: solid; border-width:1px; border-color:#cccccc; 
	margin-right:2px; color:#000000; margin-bottom:8px;
	background: url(/img/css/layout/icon_info.gif) left center no-repeat;
	background-color:#ffffff;
}

Or any other layout for that matter. The images are small icons and also optional, of course πŸ™‚

Transient flash messages

You can also put flash messages on top that are not stored in session but Configure (for this request only).
This can be useful if you don’t intend to redirect and don’t want them to show up if that happens.

From your controller:

$this->Common->transientFlashMessage('This page is currently being redesigned', 'info');

Or from your view ctp (even elements, blocks or the layout if it happens prior to the flash message output):

$this->Common->addFlashMessage('This page is under maintenance. It may be broken!', 'warning');

Output in a specific order or only specific types

You can filter the output, both in order and types:

// Using Common helper
echo $this->Common->flash(array('warning', 'error'));

In this case it would only output the warning and error messages, in this order (Usually the order is "error, warning, success, info").

Details

For details please the Wiki page.

Outview

It would probably be nice to add element support at some point. This would allow an easier approach to customization of those messages.

There are also quite a few CakePHP core tickets open regarding flash message enhancements – see this or this which might even lead to an own component for it some day.
But until then my approach will still be used in all my xx apps πŸ™‚
So in 3.x there will be a Flash component and Flash helper to provide a clean way to produce flash messages. If they provide the same features my implementation currently does, will have to be investigated. But it will use templating which will sure be nice.

Update 2014-12

A FlashComponent and FlashHelper now replace the current way of handling flash messages. This helps to ease migration to 3.x at some point, as there this is also extracted from the Session classes.
As syntactic sugar you can directly use the types as methods now:

// Normal
$this->Flash->message('Yeah', 'success');
// Possible now
$this->Flash->success('Yeah');
// Same with
$this->Flash->warning('Careful');
$this->Flash->error('o_O');
$this->Flash->info('Some information');

2015-01 CakePHP 3

The documentation for 3.x can be found in the repo itself now: Docs. Check it out!

2016-10 CakePHP 3.3+

If you don’t want to have an own type "auth" for the AuthComponent flash error message, simply load your AuthComponent like this:

$this->loadComponent('Auth', 
    ['flash' => ['key' => 'flash', 'element' => 'error']]);
 
3 Comments

Posted in CakePHP