RSS
 

Emoji and CakePHP

11 Nov

You might think: Why do I care.
Well, you would be surprised – but pretty much any website with at least one form on the page will be affected here.
So anyone with even just a basic contact form should think about applying the following changes.

A wile back I got reports that one of my older websites has problems with emoji smileys and alike.
When browsing it with a mobile device and entering a native mobile smiley the emoji and all the following text would be lost when saving.
This was especially annoying with a longer conversation that you then have to retype – many people will not know why the text was truncated and will even run into the same issue again and again.

What’s the issue?

It turns out that native emoji are 4 byte characters, and the normal MySQL database, its tables and fields were all "just" utf8_unicode_ci and only supported less than that.

How to fix it?

First of all: BACK UP YOUR DATA!

With MySQL 5.5+ it is now possible to convert everything to utf8mb4_unicode_ci and then it will safely accept all those characters now, as well.

In your database.php in 2.x or app.php in 3.x you need to change the encoding from utf8 to:

'encoding' => 'utf8mb4',

Then you must convert your database, the tables and all their fields to to the new collation.
You can use my Setup plugin to automate this:

bin/cake Setup.DbMaintenance encoding

But with dry-run param -d you can also just output the SQL commands and manually run them via Migrations plugin, for example.

If you now paste emoji characters and store them, they should be visible just fine on your website now.
Also make sure emails like via contact form are working properly.

Troubleshooting

You can run into the following issue:

SQLSTATE[42000]: Syntax error or access violation: 
1071 Specified key was too long; max key length is 1000 bytes

In that case make sure you modify your fields here. With the 4byte length some fields cannot be varchar(255) anymore, but must be a bit less (250) in oder for the convertion to work out. You could also look into "innodb_large_prefix" and options to increase the limit here that way instead.

Can I use the 3.x shell for 2.x?

Sure, with a trick. See use-3-x-migrations-for-your-2-x-cakephp-app on how to provide 3.x shell access and access to my Tools and Setup shells even for 2.x apps.

Tips

If you have a larger database and tables > 1GB you definitely want to put the site into maintenance mode and do this during the night or a very inactive time-frame, as it will need quite a few minutes to complete.

It will also increase your DB size, in my case from 2.1 to 2.5 GB. But that is worth it if it does not silently truncate data anymore.

Or just use Postgres

It turns out that Postgres natively supports emoji and alike, without any need for mb4 modification.
So maybe, if you still use MySQL it could also be time to switch

Further links

0.00 avg. rating (0% score) - 0 votes
 
No Comments

Posted by Mark in CakePHP, MySQL

 

CakePHP and connecting to GitHub API

22 Oct

I had to write a small tool to automate releasing certain GitHub repositories, and for that to authenticate I had to connect to GitHub API.
The integration wasn’t super-easy as there was no documentation yet on how this could be done. But I finally figured it out and want to share it.

HybridAuth plugin

I first introduce the plugin pretty quick I chose to use as authentication piece.
HybridAuth is maintained by a CakePHP core developer and bridges the original HybridAuth implementation into CakePHP. That library aims to "act as an abstract API between your application and various social APIs and identities providers". Out of the box it provides quite a few very popular services to connect to.

Getting started

I did install the plugin as documented, I also made sure the Migration file for it has been included because
we do need a "social_profiles" table here.

Then I connected the Users and SocialProfiles table:

    /**
     * @param array $config The configuration for the Table.
     *
     * @return void
     */
    public function initialize(array $config) {
        parent::initialize($config);
        ...
        $this->hasMany('ADmad/HybridAuth.SocialProfiles');
        EventManager::instance()->on('HybridAuth.newUser', [$this, 'createUser']);
    }
    /**
     * @param \Cake\Event\Event $event
     *
     * @throws \RuntimeException
     *
     * @return \App\Model\Entity\User
     */
    public function createUser(Event $event) {
        // Entity representing record in social_profiles table
        $profile = $event->data()['profile'];
        $username = substr($profile->profile_url, strrpos($profile->profile_url, '/') + 1);
        $user = $this->newEntity(
            [
                'username' => $username,
                'email' => $profile->email
            ]
        );
        $result = $this->save($user);
        if (!$result) {
            throw new \RuntimeException('Unable to save new user:' . print_r($user->errors(), true));
        }
        return $result;
    }

I used the "profile_url" data to automatically generate the same user on my website.
Since the login was only allowed via GitHub login, there was no change of collision.

Then I made sure the HybridAuth authentication adapter is added to the list of components in the AppController:

    /*
     * @return \Cake\Network\Response|null|void
     */
    public function initialize() {
        parent::initialize();
        ...
        $this->loadComponent('TinyAuth.Auth', [
            'authenticate' => [
                'Form',
                'ADmad/HybridAuth.HybridAuth',
            ],
        ]);
    }

I also modified the login according to the documentation.

And finally I just needed a link in the navigation menu in the case the user is not logged in yet:

echo $this->Html->link(
    'Login with GitHub',
    ['plugin' => false, 'prefix' => false, 'controller' => 'Account', 'action' => 'login', 
        '?' => ['provider' => 'Github', 'redirect' => $this->request->query('redirect')]
    ]
);

Note that the "redirect" query string is only necessary for CakePHP 3.4+ when the session is not used anymore for remembering the location to redirect to after login. And also note that at this point only a "dev" branch of the plugin supports the 3.4+ version yet.

Figuring out the configuration

Now that was the most difficult part. With a lot of debugging I found out that since the GitHub provider is not one of the core ones I need to provider wrapper path and class here:

    'HybridAuth' => [
        'providers' => [
            'Github' => [
                'enabled' => true,
                'keys' => [
                    'id' => env('AUTH_ID_GITHUB', ''),
                    'secret' => env('AUTH_SECRET_GITHUB', '')
                ],
                'wrapper' => [
                    'path' => ROOT . '/vendor/hybridauth/hybridauth/additional-providers/hybridauth-github/Providers/GitHub.php',
                    'class' => 'Hybrid_Providers_GitHub'
                ],
                'scope' => 'user:email,repo'
            ]
        ],
        'debug_mode' => false,
        'debug_file' => LOGS . 'hybridauth.log',
    ],

Note that I also set custom "scope" permissions here, you can however here leave that out or add more.

Small tweaks

I didn’t want to use the the plugin controller action to authenticate, but my own in AccountController (in order to execute a few custom things upon login).
So I just overwrote the hauth_return_toURL:

'hauth_return_to' => [
	'controller' => 'Account', 'action' => 'authenticated', 'plugin' => false, 'prefix' => false
]

Testing

Yeah, ok, here I did cheat.
The hybridauth library has a little flaw that makes it difficult to connect to CakePHP as plugin: It always forces the session to be started right away. Especially when testing the controllers now this can be super annoying as it throws ugly warnings:

..Warning Error: session_start(): Cannot send session cookie - headers already sent by 
(output started at phar:///home/vagrant/Apps/.../phpunit.phar/phpunit/Util/Printer.php:134) 
in [/home/vagrant/Apps/.../vendor/hybridauth/hybridauth/hybridauth/Hybrid/Storage.php, line 20]

So I just added the adapter when not in CLI mode:

if (PHP_SAPI !== 'cli') {
	$authenticate['ADmad/HybridAuth.HybridAuth'] = [
		...
	];
}
$this->Auth->config('authenticate', $authenticate);

Tests are green again πŸ™‚

In short

All in all HybridAuth is a great CakePHP plugin to connect this HybridAuth library and any OpenID and OAuth authenticated service to your application.
Give it a spin!

Besides the here mentioned GitHub provider I also managed to use Facebook/Google sign-in this way in another app.
Basically all apps, if for technical users or more a social network, can benefit from such a one-click login as it really takes away the pain of double-opt-in registration forms and alike.

0.00 avg. rating (0% score) - 0 votes
 
No Comments

Posted by Mark in CakePHP

 

CakePHP Tips Fall 2016

18 Oct

Always use proper version check method

Be aware of version checks that are using the inferior and even wrong </> comparison.
This can lead to wrong conditions with higher minor versions.

I had this in my code from the early 2.x days:

if ((float)Configure::version() < 2.4) {
    ...
}

With the upcoming 2.10 release this will now suddenly start to return true. Luckily my 2.next tests already revealed that and I was able to fix it up properly:

if (version_compare(Configure::version(), '2.4') < 0) {
    ...
}

This will now work in all minors of 2.x.

The same is true for 3.x, of course, even though we are "only" at 3.3.6 at this point in time yet.

Order your flash messages semantically

You can use the Flash plugin to enhance your flash messages a bit.
It works out of the box with your current setup, but adds a few useful things on top:

  • Prevent size and output issues when running into a redirect loop boiling up messages in the session (>> 50) by limiting the stack to 10 by default.
  • Use ordered output (error first, then warning, success and lastly info). Errors should be visible first in case multiple types are to be displayed.

Switch away from basic log files

I would switch away from basic log files to database – even if it is just a simple SQLite one like DebugKit uses.
This way you can more easily navigate and filter through them.
Check out the DatabaseLog plugin which can do exactly that in just a few minutes of configuration.
For not just personal fun apps it is advised to look into a bit more sophisticated approaches, like using Monolog and services like NewRelic to write to. But for smaller projects it can be enough to have a small admin backend to filter through the logs, especially error and warning types.

Keep your controller code lean

That specifically includes all those verbose allow statements to make certain pages public:

use Cake\Event\Event; 
...
 	/**
	 * @return void
	 */
	public function beforeFilter(Event $event) {
		parent::beforeFilter($event);
		$this->Auth->allow('index');
	}

See TinyAuth which since recently allows to more cleanly declare this with a single INI file in your src/config folder.
The complete code above goes away for every controller πŸ™‚

As long as not needed (custom dynamically made decisions based on session or alike) this kind of noise should be kept out of the controllers.
This makes it also possible to modify the public access to actions without the need to be a developer. The next step could be to make the input for TinyAuth database-driven and alike.

0.00 avg. rating (0% score) - 0 votes
 
No Comments

Posted by Mark in CakePHP

 

Use 3.x Migrations for your 2.x CakePHP app

03 Oct

In this post I reveal one of my tricks on how to leverage 3.x power in a legacy 2.x project.

You might have already read on how to use some of the splits, like the ORM, in 2.x projects.
Today I want to talk about migration as a topic.

Status Quo

I do have to maintain two remaining CakePHP 2.x apps that have been too large to just upgrade yet.
And time and budget was not on my side so far.
In 2.x there was also not a real powerful database migration tool available so far.

Let’s use the 3.x Migrations plugin

We can use the Migrations plugin quite easily in all 2.x apps do all database modification this way.

First we create a subfolder in your 2.x root folder, let’s call it /upgrade.
This will contain a standalone 3.x app including the Migrations plugin as dependency.
In my case the composer.json looks like this:

	"require": {
		"cakephp/cakephp": "~3.3",
		"cakephp/migrations": "~1.6",
		"cakephp/bake": "~1.2",
		"dereuromark/cakephp-setup": "dev-master",
		"dereuromark/cakephp-tools": "~1.1"
	},
	"require-dev": {
		"cakephp/debug_kit": "~3.2"
	},

Since I include cakephp/bake, I can also leverage the Bake plugin to generate the necessary migration file.

Any time I need a new migration file I simply go to the subfolder and use the 3.x shell:

cd upgrade
bin/cake bake migration CreateArticles ...

Of course you can now modify it further and once complete commit this file into version control.

On the server your deployment script just also needs to contain the following lines then to fully automate it:

cd upgrade
chmod +x bin/cake
bin/cake Migrations migrate
cd ..

Once you upgrade to 3.x you can move all migration files to the actual place in your app, remove the subfolder and simplify the deployment lines to just the single command πŸ™‚

At least I now have to only remember one way to do migrations, for all 3.x and the old 2.x apps. And I can benefit from all recent improvements in those plugins even in those old apps.

Upgrading from existing 2.x migration tool?

You might be using a 2.x tool like this already, but upgrading to the state of the art 3.x Migrations plugin hotness is not a problem here, either.

Just create a dump of the current schema, put it into an SQL file and include that in your first Migrations file:

public function change() {
	$sql = file_get_contents('dump.sql');
	$this->query($sql);
}

Make sure you mark it as migrated, so it is not accidentally executed again.
As alternative you could build in a switch to auto-detect if one of the tables already exists:

$exists = $this->hasTable('users');
if ($exists) {
	return;
}
...

Tips

Deploy admin user along with the schema

In some cases it can make sense to provide a basic admin login along with the first initial migration:

$data = [
	[
		'username' => 'admin',
		'email' => 'admin@example.com',
		'password' => '...' // Must be changed right afterwards
	]
];
$table = $this->table('users');
$table->insert($data)->save();

For more, it is advised to leverage the seed functionality the Migrations plugin ships with.

Further goodies

As you can see, I also included my Setup and Tools plugin, which in v3 also contain very useful and powerful tools for database maintenance and alike. I can now also leverage them and do not have to backport everything to 2.x anymore.
The same would be true for any such plugin and will help you save time for other things if you can focus on development for the 3.x branch only.

Bottom line

Every pre-3.x project should definitly have a subfolder which runs an up-to-date CakePHP 3.x shell including all useful and required plugin shells.

Feel free to share your ideas and experiences on advancing slowly towards 3.x as comments.
See my old post about it for some more details on how to share the credentials, so you can keep them DRY in your main app config.

0.00 avg. rating (0% score) - 0 votes
 
No Comments

Posted by Mark in CakePHP

 

Chronos – Let there be time

12 Aug

In PHP most know of DateTime class to handle date and time.
At least with more modern PHP versions it is now not advised anymore to use the plain old date() and time() functions.

The use cases – especially with a more global world – these days more often include correct time zone handling as well as more robust
delta handling. But using objects also means you have to be more careful about (accidentally) modifying the original date when you are creating a new one from it.

Until now the de-facto standard pretty much was Carbon – as we all know.
It wrapped the DateTime object and applied some necessary bugfixes as well as a lot of useful enhancements like better object oriented access for reading and writing.

We at CakePHP also wanted to start using it, faced quite a few issues though at the time.
One was, that it seemed unmaintained over months of time – often times with critical or at least major bugs not being fixed.
There was also the problem that there was no (and still not is) fork or version for a more modern PHP 5.4+ approach. We actually wanted and needed PHP 5.5+ support due to a lot of necessary enhancements of date time handling in PHP, more to that later.
So even after approaching them multiple times, trying to offer a helping hand here, not much changed.

The solution for us then was the only viable one on the table: We need to create a clone of it, and start maintaining it ourselves.
Chronos as modern and future proof stand-alone library to handle date and (date)time.
As a side-effect we were able to also implement better interfaces around it and could leverage all the new PHP features.

And as of this week, the Chronos library is officially marked as stable 1.0.0 πŸ™‚

Main differences and improvements

  • Implements ChronosInterface for proper typehinting, e.g. in methods
  • DateTime and Date (no time) handling separately per use case
  • Immutable by default for cleaner coding and less errors
  • Correct some faulty standards (ISO code violations) and behavior (difference calculation)
  • No external dependencies

Mutable vs Immutable

That topic is present throughout many layers of data handling, but with objects being passed around by reference, this is especially important around
data manipulation inside your business layer.
Using mutable by default means that you could easily modify a DateTime object (or Chronos in this case maybe) by accident.
You could have added a day to check if that following one is still a weekday, but at the same time this modification than accidentally back out of the method and down the chain of method invocations. The next method then uses the altered datetime and so on and so on.

// Bad practice - and doesn't work with immutable objects
$datetime->addDay(1);
$this->doSomething($datetime);
return $datetime;
// Better to never touch the original object - this works like you'd expect
$datetime = $datetime->addDay(1);
$datetime = $this->doSomething($datetime);
return $datetime;

Shimming buggy PHP core behavior

Intuitively, if you add months instead of specific days to a date, you would expect this to be "month-exact", not "day-exact".

$dt = new DateTime('2015-01-31');
$dt->modify("+1 month");
echo $dt->format("Y-m-d H:i:s");   //2015-03-03 00:00:00

Clearly, this overflows in unexpected ways.

So Chronos actually gives you addMonths()/subMonths() that actually work as desired:

$dt = new Chronos('2015-01-31');
$dt = $dt->addMonths(1);
echo $dt->format("Y-m-d H:i:s");   //2015-02-28 00:00:00

To get the former PHP behavior back, you can explicitly use addMonthsWithOverflow()/subMonthsWithOverflow() methods. Not that is is ever useful or recommended πŸ™‚

Testing and fixating time

Everyone knows those one second issues when writing tests and (date)time. Sometimes tests fail because the time for now() jumped to the next second.

When writing unit tests, it is helpful to fixate the current time. Chronos lets you fix the current time for each class. As part of your test suite’s bootstrap process you can include the following:

Chronos::setTestNow(Chronos::now());
MutableDateTime::setTestNow(MutableDateTime::now());
Date::setTestNow(Date::now());
MutableDate::setTestNow(MutableDate::now());

This will fix the current time of all objects to be the point at which the test suite started.

Usage

The Chronos API offers a very fast and intuitive way to work with datetime.

Let’s say you want to find the next Tuesday, if the current one is not already one:

$dt = new Chronos('2015-01-31');
if (!$dt->isTuesday()) {
    $dt = $dt->next(ChronosInterface::TUESDAY);
}

Quite convenient are also the checks to find out whether a date is in the past or the future:

$dt->isPast();
$dt->isFuture();

Of course, you could also use a more verbose way with gt()/lt() and a current "now" datetime.

Check out the official chronos docs for how to use it in general.

Usage in frameworks

Usually, frameworks should be able to switch inside DB layer from DateTime or Carbon to Chronos easily.
In CakePHP for example the type conversation is setup in the bootstrap, and it already uses the immutable Chronos objects by default:

// bootstrap.php
/**
 * Enable immutable time objects in the ORM.
 *
 * You can enable default locale format parsing by adding calls
 * to `useLocaleParser()`. This enables the automatic conversion of
 * locale specific date formats. For details see
 * @link http://book.cakephp.org/3.0/en/core-libraries/internationalization-and-localization.html#parsing-localized-datetime-data
 */
Type::build('time')
    ->useImmutable();
Type::build('date')
    ->useImmutable();
Type::build('datetime')
    ->useImmutable();

The underlying classes here extend Carbon and so in the ORM all ingoing and outgoing datetimes are Chronos objects.

0.00 avg. rating (0% score) - 0 votes
 
No Comments

Posted by Mark in PHP

 

Use isset() and empty() wisely

01 Jun

This is another part of the series "How to write better PHP code".

Sometimes it is the small things than can make all the difference. In this case simply using the correct way of checking
for variable content can avoid quite a few bugs in the long run.

What is the issue here

Very often people abuse empty()/!empty() and isset()/!isset() where they shouldn’t. I have seen this in many frameworks,
and in retrospective also in my own code for quite some time.

The main problem is that it does two things at the same time:

  • Checks if that variable exists at this point in time, if not silently bail early
  • If it exists it also checks of the content is not empty or set

Now, that sounds like "so who cares" – but it in fact has quite some impact in the correctness of code, especially when refactoring is involved.

Concrete issue

There was an actual bug in the CakePHP framework because of this.
The class attribute $compiledRoute was renamed, but a single use of that one was checked with !empty(), so that part of the code inside this check became unreachable and dead forever. Since this one piece of code didn’t have a test for it, it also remained dormant for a very long time.

Buggy behavior with methods

Since PHP 5.5+ it is now even possible to check expressions and more, so !empty($this->foo()) would be possible.
This should never be used though as this can have side-effects depending on the implementation of the class and is absolutely not necessary.

So what to do

The basic rule is: Always prefer non-silent checks.
Also always try to compare with === checks and expect the variable or attribute to be declared at run-time.

Only ever use empty()/isset() if that is not possible, that means if there is a chance that the variable or attribute could be not set at this point in time.
Same goes for array keys and alike.
But for variables and attributes you might want to ask yourself it that is maybe not a code smell after all if you do not know that for certain.

Luckily IDEs these days provide a lot more help than a few years ago, but there is still lot of room for error.
Let’s look into more concrete examples now.

Examples

Method arguments

public function read($key, $context = null) {
    ...
    if (!empty($context)) {
        // Do something
    }
    ...
}

What happens now if you refactor $context in the signature, but forget the one inside the if check?
Using this approach would have 100% identical behavior while making sure you couldn’t have forgotten it:

public function read($key, $context = null) {
    ...
    if ($context) {
        // Do something
    }
    ...
}

Ideally you would also look into more strict checks if applicable, but this is already way better than before.

Class attributes

protected $foo;
public function read($key) {
    ...
    if (isset($this->fooo)) { // Notice the typo
        // Do something (oh no, this will never happen now)
    }
    ...
}

While coding, we made a typo and never notice it. If you also don’t bother to write a test for this case, you might only discover it very late in the process, if at all.
So it should look like:

protected $foo;
public function read($key) {
    ...
    if ($this->foo !== null) {
        // Do something
    }
    ...
}

Renaming only one of the two would immediately trigger an error and the developer gets notified about his coding mistake.

Array keys

protected $config = [
    'recursive' => false
];
public function read($key) {
    ...
    if (!empty($this->config['recursive'])) {
        // Do something
    }
    ...
}

We can see that the key must exist. If not, we screwed up. So why using the wrong check that would only cloak spelling mistakes.

Better code like:

protected $config = [
    'recursive' => false
];
public function read($key) {
    ...
    if ($this->config['recursive']) {
        // Do something
    }
    ...
}

Optional array keys

public function read($key, array $options = []) {
    ...
    if (!empty($options['recursive'])) {
        // Do something
    }
    ...
}

In this particular case the recursive option is optional, thus the key could be empty here.
This would be OK.

One way to maybe improve this would be to merge the $options array with some $defaults which contain all required keys, then you can also use the proper non-cloaking checks again.

More examples

Don’t wrap methods, especially for "object or null" return values:

// Bad
if (!empty($this->getSomeObjectOrNull()) {}
// Good
if ($this->getSomeObjectOrNull() !== null) {}

I also often see the redundancy that makes no sense:

// Bad
if (isset($x) && !empty($x)) {}
// Identical (but maybe we also don't need the !empty here?)
if (!empty($x)) {}
// Better (maybe even strict checks?)
$x = null;
...
if ($x) {}

Usually variables should not just sometimes exist. Declare them as null first if you conditionally populate them somewhere.

Bear in mind here, that 0 and '0' would not work with either way and in that case you need to be strict in your checks anyway.
There also many buggy comparison cases around the 0 integer and a reason more to try to be more strict in general wherever possible.
That includes in_array() by the way, one of the functions probably most (framework and project) bugs have been discovered the last months due to this implicit casting issue leading to false positives.

Further reading

There was an RFC discussion in CakePHP around the same issue, as direct result of a bug that would never have been there if the above was applied properly.

Please share your thoughts on the topic as comments.

5.00 avg. rating (94% score) - 1 vote
 
No Comments

Posted by Mark in PHP

 

Developing CakePHP 3+ Plugins, it’s fun!

29 Jan

It is fun – and it should be!

The days of CakePHP 2 plugins and how difficult it was to actually develop plugins are over.
Back in the days (OK, I still have to do it once in a while), there was even an app required to test a plugin. Since you didn’t want to have a boilerplate app for each plugin, you usually worked in your actual app. So you had cross contamination from that messing up your tests and stuff. Really annoying.
The only thing I am still missing once in a while is the web test runner, as it speeds up things for some use cases (usually with browser related output).

While most of the concrete examples are about plugin development for CakePHP 3, the main ideas apply to all library code you write. And if you are a developer for other frameworks, the same principles apply, only the concrete implementation might differ. So you could skip the "real story" part.

Well, but now to the fun part I promised in the title.

CakePHP 3 plugin development – real story

My goal was simple: Developing a simple plugin for hashids support in CakePHP in a few hours.

The CakePHP plugin docs mentioned a few basics, but in the following paragraphs I want
to go into more concrete details.

How do you start?
I started by creating a fresh GitHub repo "cakephp-hashid" and cloning it (git clone URL).

Then I added the boilerplate stuff like composer.json and gitignore file. You can either copy and paste from existing ones,
or even bake your plugin code (cake bake plugin Name) and move it to your stand-alone plugin.
Keeping in the app is also possible, but I prefer to keep it outside and develop it test driven until it is in a working state.
This way you are usually faster. TDD – test driven development – actually helps to speed up development, and you get tests with it for free.

Now it was time to set up the behavior code and the constructor setup as well as a test file.
With php phpunit.phar I already got immediate results of the initial work, and could fix those with almost zero overhead.
As soon as I added more use cases, especially with some config options and edge cases, I quickly saw where things were not working as expected.
But while getting those to run, I also saw if I broke the existing already working tests. Big help.

Once I got everything in a beta usable state, I pushed and published the repo.
After some documentation (!) in the README as well as Travis/Packagist enabling, I already composer required it in my Sandbox project and started to use it "for real".
When I got it all working in live mode, I felt convinced enough to release a 0.1 beta tag.

The cool thing that coveralls (or codecov.io etc) told me then:
I got 95% code coverage out of the box without writing additional tests (I did write them parallel to the code as "pseudo app test frame").
Simply because when I wrote the plugin code and tests, I already tried the different options available, running into all possible code lines by design.

So now I added more features over the next 1-2 hours, released 2 more tags and after a few days it is now time for the official 1.0.0 release.

Here is my sandbox demo code for this plugin, by the way: sandbox3.dereuromark.de/sandbox/hashids.

In parallel I started even a 2nd small plugin Markup, which took my only half the time even because now I was already quite fast in setting up the boilerplate stuff. Here I also tried to keep it extensible for use cases of future plugin users.

So overall I invested a few hours total to have two easily maintainable plugins that are open for extension, but by default suit my needs. Try it yourself, you will see that this way it really is not too hard to develop and publish such a plugin.

(Plugin) coding tips

If you develop a plugin for the first time, take a look at the existing ones listed in the awesome-cakephp list.
They might give you some insight in how things can look like. How we add a bootstrap for testing, how a Travis file looks like etc.

For test cases it also never hurts to take a look into the core test cases.

Plugin vs. core feature

This issue comes up every week basically. For me, beginning with CakePHP it was difficult to tell what should be covered by the core and what should stay as community plugin. I had this idea that every use case must be supported by the framework itself. Over time, it become more and more clear to me that a framework itself should stay lean and focus on the majority of the use cases and maybe provide a way to make it extensible for those edge case.

As a logical conclusion some of the CakePHP core functionality has been split off into it’s own repositories, like Acl, Migrations, Bake, Localized.
Not all of the users need those additional tools, in fact almost no one used Acl, and you only need Bake for development.

The positive side-effect here is that the iterations of these plugins can be independent from the core development, making feature development faster as there is no core release for the whole package necessary anymore.

Ask yourself the following questions:

  • Does the majority of devs need this functionality, and if so, this specific implementation of it?
  • Is it beneficial for both core and plugin to keep it standalone for maintenance and extendability reasons?

There will be more questions you can ask if you continue reading the next chapters about more generic principles.

Try to follow coding and package principles.

With CakePHP 3 we can finally adhere more correctly to some very basic coding principles. Most of you might know (or at least heard) about SOLID and Package Principles.
They following tips go into more detail what it means for our CakePHP plugins.

Coding principles (SOLID)

SOLID principles

The 5 well known SOLID principles can help us deciding how to best code our plugins and libraries.

Single responsibility principle (S)

Try to keep your classes simple and short, doing one specific task.
Take a look into my Geocoder behavior.
It does not actually geocode, because that is the task of a specific class. To avoid it doing too much, the behavior only wraps this Geocoder class and forwards calls to it. This way the only responsibility of this behavior is to help the model layer (usually a Table class) to geocode the entity data, while the single responsibility of the Geocoder class is to perform this geocoding task by talking to an API.
The additional advantage is that we can also use the library class as standalone, so we might want to provide a GeocodeShell, for which we most certainly don’t want to use behavior to encode a simple input string.

Open/closed principle (O)

Your code should be open for extension, but closed for modification. You will most likely never be able to guess and support all use cases for your code out of the box. Often times people will have come up with new ways to use your plugin.
So at the one side you do not want to have to change your code for every possible scenario. But if it was possible to support a lot of extensions out of the box, why not doing this?

If we are using dependencies in our classes, we do not want to rely on a specific class dependency, but an interface.
This might be not so important to you as plugin designer, but as soon as people use it, they might want to change maybe how a specific implementation of your dependency works.

Lets take a look at the [Slug plugin][(https://github.com/UseMuffin/Slug). It shows how you can simply set a default for 'slugger' => 'Muffin\Slug\Slugger\CakeSlugger', but it would allow any other slugger implementing the SluggerInterface. Instead of providing and versioning all possible sluggers (and their potential dependencies) itself, it allows another package to contain a user-specific slugger
to use.

So always try to not contain specific switch statements or if/else blocks. Instead, always implement against a generic interface that allows for exchangeability. This is especially important for type-hinting constructors and methods.

Here, the implementing classes just need to provide a slug() method and it will work just fine even with some BinarySlugger πŸ™‚

Liskov substitution principle (L)

Every subclass or derived class should be substitutable for their base/parent class. So make sure you make don’t widen the input/constructors, but keep them the same or narrow them.
You can always become more strict, but not less. Interfaces also help with that, as they make sure that at least those common methods have been provided.

Interface segregation principle (I)

If you create one interface containing too many different method stubs, often times you limit the possibilities of implementation.
Often times those classes can be grouped by API or Non-API, and in either of those cases need only a subset of the interface methods.
In this case it will most likely make sense to have two specific interfaces for each use case, this allows the sub-parts of your code to only rely on those relevant methods they care about.

Dependency inversion principle (D)

Ideally, we always enforce class dependencies via constructor argument commonly known as "Constructor Dependency Injection". When allowing to exchange the used class, we should respect that.

So in our case, we allow a closure to be passed into our plugin library class config:

$geocoderClass = $this->config('provider');
if (is_callable($geocoderClass)) {
	// Use the injected closure
	$this->geocoder = $geocoderClass();
	return;
}
// Manually create class
$this->geocoder = new $geocoderClass();

You never know if the class your plugin users want to use require some constructor dependencies on their own.
The DI principle should be possible for them to use, too, to fully respect the Open/Close Principle from above.

They can now have their own HTTP adapter passed into the Provider class:

$config = [
	'provider' => function () {
		return new \Geocoder\Provider\FreeGeoIp(new \Ivory\HttpAdapter\CakeHttpAdapter());
	}
];
$this->Geocoder = new Geocoder($config);

As with the first principle, it is also important for DI that if you type-hint methods, class properties and alike, always try to use the Interface, not a concrete class.

Note that there is a second way to pass class dependencies usually referred to as "Setter Dependency Injection".
This should be avoided for required classes and only be used for optional parts, if any.

Package principles

Some of the 6 commonly known package principles can also be quite useful for our decisions.

Reuse-release equivalence principle – consider the scope

It is very much possible that whatever plugin you are going to write, the functionality itself could be interesting to other PHP applications.
With 2.x it was hard to provide it both ways, with 3.x this is now as easy as it gets.

A good example is Mark Story’s AssetCompress plugin.
Back in 2.x this was a full blown CakePHP-only dependency.
He realized that most of the code behind it could very well be useful to other PHP projects, though.
With CakePHP 3 it was finally possible to move the code into a standalone lib called MiniAsset.
All that was left of the plugin code was to function as a CakePHP bridge. This enables the actual library to be used by everyone whereas the framework users could use the plugin.
r
So also check your existing (plugin) code for this maybe.

Common-reuse principle (CRP)

Code that is used together, should be ideally in the same package. Micro-splitting each class into it’s own plugin might not always be wise. It also increases maintenance time/costs.

So if you have a Geocoder Library class talking to the API and a Geocoder Behavior using this Library to populate entities, you would always use them together, or just the library class.
Splitting them into own plugins does not sound feasible here, as the default case sees them used together.

Common-closure principle (CCP)

A package should not have more than one reason to change. So when you put too many different classes or even multiple different dependencies into a plugin/lib, you might have to major bump for each of those separately, creating a lot of annoyance for the users only using the unrelated part of code.
Bear that in mind when packaging. Packaging a lot together eases maintenance time/costs, however. So you need to find a good balance here (also regarding CRP principle) for the maintainer(s) as well as the package users.

Most probably know my Tools plugin, which was started way back in 2006 with SVN and no github/composer.
At that time it would have been super painful to support multiple plugins via SVN externals and no release/version locking really.
Thus one monolith class collection. Over the time, with GitHub and more people using it (not just me anymore), it became clear this approach is outdated and troublesome. Most of the time people use only a small subset and have to get all irrelevant class changes on top.
So with CakePHP 3 already becoming alpha/beta I started to split off some of the chunks that really deserve their own plugin namespace.
TinyAuth, Geo, Ajax, Rss and Shim have been created.
Now everyone using TinyAuth or Rss, for example, will most likely use all those classes together, while inside Tools it would have been 1% of all the rest.

I will further split out stuff in the future, wherever this makes sense regarding to these two first packaging principles. But it would also be impractical to have to maintain 150 new plugins for each small stand-alone class used somewhere at some point. So a good balance is the key again.

Package coupling (ADP, SDP, SAP)

  • Prevent cycling dependencies
  • Change is easiest when a package has not many dependencies (stable), so make sure if it does those are also not blocked for change.
  • Stable packages ideally have a lot of abstraction (interface, …) exposed to the depending parts so their stability does not prevent them from being extended.

Note that the term "stability"/"instability" isn’t evaluative. It simply refers to the dependencies of a package here.

A quote about stability of classes:

Typically, but not always, abstract classes and interfaces are not dependent on others and tend to be more stable as against concrete or implementation classes. This is because the abstract classes or interfaces typically represent the high level design and not the implementation, e.g, an interface called Logger will be more stable than classes FileLogger or DBLogger.

ifacethoughts.net/2006/04/15/stable-dependencies-principle

So in both cases (class vs package level) stability means something slightly different but in the end sums up to "less reasons to change".
By trying to create and use packages and classes in the direction of stability usually is the best approach and will keep necessary changes at the endpoint (usually your app) at a minimum, as well as for most parts (plugins) in between.

Releasing

Don’t forget to release your code with tags following "semver".
You can start with 0.x and once you feel comfortable that it is fairly stable, release a 1.0.0.
New functionality usually goes into minor releases, bugfixes in patch-releases. A BC break warrants a major version jump.

Framework Semantic Versioning

With releasing plugins for a CakePHP version strict semver can be somewhat confusing, though (1.x/2.x here is for 3.x there, 3.x+ is for 4.x there, etc).
One more severe problem with that is that once you released a new 3.x framework compatible version you cannot major bump your 2.x code, as there is no number left in between. You have to break semver, or do some other workaround using composer-constraints. Some might even suggest to use a new repository for the 3.x compatible code etc.
Most of the time people are just afraid of major bumps and often use a minor one to introduce larger breaking changes.
It seems like here the design itself does not fit to the use case and misleads the maintainer to do bad things (talking from experience).
I think there is a better approach.

Something that could be considered "framework-semver" is the following approach I have seen occasionally so far:

x.y.z

  • x: CakePHP major version
  • y: Plugin major version
  • z: Plugin minor/patch version

Examples:

  • 2.3.4: CakePHP 2.x, Plugin 3.x for this major, Minor/Patch v5
  • 3.1.0: CakePHP 3.x, Plugin 1.x for this major, First Minor

So the first number of both core and plugin matches.

In my book this makes it way more clear as the plugin itself cannot live without the CakePHP core dependency and at the same time has to be compliant to each of those different major versions.
So in all this makes more z bumps, and occasionally an y bump. But that is OK.
It also requires you to look the y version then "vendor-name/plugin-name": "1.2.*", as every y bump could break BC.

In theory you could also use a 4th digit, the actual patch version: x.y.z.patch, e.g. 2.3.4.1. This would make it fully semver in itself again, but is usually not needed, as new functionality that is BC and bug fixes are both supposed to be z compliant (otherwise it would be major plugin version bump).
Either way it solves the issue of framework dependent plugins by design, and not by workarounds.

A smilar approach was already published here, but this is not composer friendly, and since all non major bumps should be BC, there should be no immediate need for a prefixing that includes the minor version of a framework.

Note: This is just some grounds for discussion so far. If you plan on using this you should really make it clear and documented it in bold visible in the README or something.
Because by default people will most likely assume you are following default "semver". Are there any downsides so far? I would really like to have some feedback here from the whole community.

License

Most people tend to forget that they release packages that are to be used in other peoples’ (proprietary) software. That means that they cannot legally use your code unless you specifically put a license in they know they can use. An MIT license is recommended and always works.
But that requires the file to be present in the root folder and/or in the files itself.
Just having a statement in the README usually does not suffice. Just something to look out for.
It would be sad if lots of projects/companies would like to use your plugin but cannot because of trivial license issues. Get it out of the way. You will benefit from more uses because of more feedback/contributions of them, as well.

Maintaining

One thing you should consider is regular maintenance for your plugins. Make sure CI (e.g. Travis) tests pass, that issues and PRs are taken care of in a reasonable time frame etc.
If that is too much to do, you can always ask for help and co-contributors, or even hand off the repo to someone else entirely.

Ready, set, …

Then take an idea and pluginize it.

You really want to start coding your own awesome plugin now but don’t have an idea? Look into the wiki of that very same repo from above. There you can find a lot of 2.x plugins that have not been upgraded yet. You could take care for any that interest you.
Or you go through your existing code and check those classes (helpers, behaviors, libs, …) for re-usability.
Also don’t forget to add your upgraded or new plugin to that list πŸ™‚

Final notes

Some people might know DIP also as IOC (Inversion of Control).
Maybe also take a look at containerless-dependency-injection-for-services, a new article of @markstory about how most frameworks make their users potentially abuse DIC and
how better code your service classes.

Book Tips / Source Materials / Interesting Videos:

5.00 avg. rating (97% score) - 3 votes
 
1 Comment

Posted by Mark in CakePHP

 

Avoid Yoda conditions

15 Oct

After blogging about the semantic importance of void as pseudo type I feel like I should also point out the nonsense of Yoda conditions.
Those are still used quite a bit in the coding world, mainly in and around WordPress and Symfony if I recall correctly.

First of all, what is it?
It is the switching of order in conditions:

if (2 === $value) {
}

Like u say β€œIf blue is the sky”.

Sounds weird? Sure is πŸ™‚

Why do they exist?

It seems they have been invented to prevent "some" mistakes (some as in only a small subset!) when writing conditions.
If you, for some reason (…?), forget to write the second = in == it will result in a parse error for a few cases, telling you that quite early in the development process.

What is the danger of using them?

The main question about coding standards is always the same. Do we write clean and readable code for the computer or for humans.
The computer usually doesn’t care about any of those standard issues. The code functionally behaves the same.
But if humans have a hard time understanding/reading the code, that will be a huge problem for maintainability.
It also introduces a high risk of making errors.
So actually, in my many years of programming I have actually seen more mistakes made because of Yoda then without.

Think about this

if (true !== is_numeric($var)) {
}
// vs.
if (is_numeric($var) !== true)  {
}

or even

if (2 < $value) {
}
// vs.
if ($value > 2) {
}

You can easily forget to negate a value when in the not so natural order or confuse the comparison direction.

Often times I see them used only for the equality checks, not for the rest of the comparisons, and that makes it super inconsistent.

if (false === $foo) {
	...
} elseif ($bar > 2)  {
	...
}

Eliminating the root problem

It seems the Yoda fans have simply forgotten to look for the root issue they try to prevent here.
And instead of fixing that, they made a half-hearted workaround that misses out on most accidental assignments and introduces
the right of human error due to the mental overhead involved.

Actually, it gives you a false sense of security.
What is won if you still can by accident do

// Autsch
if ($variable = $otherVariable) {
    // ...
}

So let’s fix it – the right way.

Preventing inline assignment

Prevent what you want to "hack avoid" consistently and without exceptions:

// Conditional assignment is disallowed
if (($variable = $this->foo()) === null) {
    // ...
}
// This is how it's done
$variable = $this->foo();
if ($variable === null) {
    // ...

Add a sniffer to be sure

Now add a sniffer to your CS (php-cs-fixer or phpcs tool) and automatically prevent this from happening ever
(100% error free, cannot be missed by human error).
Make sure, you cannot merge until the sniffer shows a "green light".
That means = to check for only, as all other operations are not assignment operators.

And now we can add a second sniffer to check on no primitive value is the first argument: 'foo'|true|false|null|, numbers (1…x), constants and CO.

Setup your IDE

IDEs like PHPStorm can detect and warn at runtime about inline assignment:

Settings => Editor => Inspections => Probable bugs => Assignment in condition (check)

Benefit

The chance of assigning by accident went from likely to not possible.
In the meantime readability increased a lot, as the natural language makes this "natural" in the way we read and understand the conditions without having
to do further calculations in our head.

Further reading

blog.codinghorror.com/new-programming-jargon says it all.
This is also valued by some of the most important PHP open source frameworks, like CakePHP.

It is also part of PSR-2-R Additions.

Update 2015-12

I stumbled upon an article where they also relized the same thing.

5.00 avg. rating (98% score) - 5 votes
 
3 Comments

Posted by Mark in PHP

 

Return null vs return void

05 Oct

The other day I had a discussion about that, and why important open source frameworks like CakePHP use void in @return doc tags when void and null are code-wise totally identical.
So here my reasoning why it is a good idea to make a distinction:

Help the developer, not the machine

We already discovered that it doesn’t help the machine to make the difference.
So if anything, it would help the developer. So does it? Yes, it does (as with all coding standards, these things are there to make it easier for the developer and avoid human error).
In fact, most modern IDEs would light up the function like a Xmas tree if you tried to use a return void function value.

The following would be marked yellow (warning) right away:

/**
 * @return void
 */
public function doSth() {
}
/**
 * @return void
 */
public function badCode() {
    return $this->doSth(); // This would be marked yellow
}

So it helps the developer to not try to use return values of those methods.
This would not happen without the explicit @return void annotation.

See the following screenshot of what PHPStorm, for example, can now help us with:

return-null-vs-return-void

And also the other way around:

enforce-return

Bottom line:
void as pseudo type is not used to state that we return nothing/null (which is still true from a value point of view), but it is used to state that we should not expect anything (not trying to use it’s return value in any way).

Other reasons maybe

Be explicit

If you just see the following code, you might not know immediately, if someone just forgot the return type statement. If it was there from the start you would automatically know.

/**
 * @param string $param Param
 */
public function doSth($param) {
    // very long code block...
}

Be consistent

Some methods end up to not have any doc block, just because the doc block would have the return void part:

/**
 * @param string $param Param
 */
public function doSth($param) {
}
public function doSthElse() {
}
/**
 * @param string $param Param again
 */
public function doSthAgain($param) {
}

Especially if you don’t use descriptions this is quite inconsistent.

Part of PSR-5 and PHP RFC

It is also part of the upcoming PSR-5 standard.
To point out that is not just fiction πŸ˜‰
Even though they state it to be optional (omitting that type altogether would also be valid, of course).

As long term goal then, once PHP RFC void_return_type is implemented, we can easily use a script to adjust the code.
Without being explicit this will not work out. So better use the best practice approach now already and save yourself tons of work in the long run.

Automated checking

It will be easier to automate and verify this via code sniffer or other tools when explicitly set, whereas the omitting does not tell if it was on purpose or just forgotten.
Especially since it forces to code more return type aware from the beginning (see the following part on potentially hidden issues around mixed return types).

Implications to consider

This means you should also use it consistently through-out the code base. Here a few things to keep in your mind.

Whenever you return multiple types (mixed), do NOT use void, use null instead.
The reason is simple:
You are returning an object or not (null), but you are using/checking the value of it afterwards, so returning void would be a lie (or trying to check on a void return result would be "pointless").

/**
 * @param Object $object The object.
 * @return Object|null
 */
public function doSthAndGiveMeSth(Object $object) {
	if ($object->isNotValid(()) {
	    return null;
	}
	// ...
}
/**
 * @param Object $object The object.
 * @return void
 */
public function doSthWithIt(Object $object) {
    $returnValue = $this->doSth($object);
    // ...
}

This is also true for returning early inside those methods/functions.

Also always explicitly return null at the end of a "non-void" method, if nothing else is being returned.

For returning early in void methods, use the following:

/**
 * @param string $param Param
 * @return void
 */
public function doSth($param) {
	if ($param === null) {
	    return;
	}
	// ...
}

An explicit return; statement at the end of those methods is never necessary (= should not be used therefore) and implies void when omitted.

Do not use return statements for constructor __construct() and destructor __destruct(), as both per definition cannot return anything.
This is the only exception made.

Adjusting your IDE

You can adjust IDEs usually to automatically add @return void as default.

For PHPStorm go to Settings -> Editor -> File and Code Templates -> Includes (tab) -> PHP Function Doc Comment.
Replace the template with:

/**
#if (${PARAM_DOC} != "") ${PARAM_DOC}
    #if (${TYPE_HINT} != "void") *
    #end
#end
${THROWS_DOC}
 * @return ${TYPE_HINT}
 */

Other implications

Exceptions

Some frameworks use special methods that can return an object, or nothing, e.g. controller methods as actions.
In that case it can be convenient to not have to return explicit nulls all the time at the end of each action.
Maybe, in some rare use cases one could use null|void (and then Response|null|void) as a way to allow omitting the explicit returns but to show that it would actually allow checking for the returned result afterwards. The IDE won’t complain then, either.
That should definitely stay the exception to the rule then, of course.

Building fluid interfaces

Once you use use a lot of @return void you end up using their methods appropriately – which implies that you don’t use the return value at all (not permitted).
That concludes that you could, basically without breaking the API, start returning the object itself ($this as in @return self). This transforms the class into fluid interfaces which allow method chaining then.
So look for those return types in your classes and think about where those could be part of a fluent interface pattern. Might not always be a good idea, but could very well be in some places.

Further reading

Take a look at php-void-much-ado-about-nothing.

It is also part of PSR-2-R Additions.

Update 2015-11

So the void RFC has been approved. Will be part of PHP 7.1 then πŸ™‚

3.60 avg. rating (74% score) - 5 votes
 
No Comments

Posted by Mark in PHP