RSS
 

Posts Tagged ‘Validation’

CakePHP Tips – Winter 2012

01 Dec

Some more tips I gathered the last couple of weeks and want to share with the (Cake/PHP)world πŸ™‚

Jquery and CakePHP FormHelper::radio()

Usually you would use

var value = $('input[name=fieldName]:checked').val();

But since CakePHP uses name="data[ModelName][field_name]" generating the form fields you cannot just use input[name=data[ModelName][field_name]]. Two brackets inside each other rare not allowed.
So you need to do it this way:

var value = $('input[name=\'data[Default][status]\']:checked').val();

The escaped field name will make it work again.

How to dynamically overwrite the error message

The most important fact for me is: This was the reason I finally figured out how to override the error message from within a custom validation method.
I played around with it and with some help from stackoverflow/irc I solved the problem I had not been able to solve so far. So with that I want to start.

Imagine a custom rule that calls a webservice or another class which then could throw an exeception. We want to catch the message of it and display it instead of the default message "Something went wrong" πŸ™‚
It is actually pretty easy, just do not return false but the error string itself:

public $validate = array(
		'url' => array(
			'validateUrl' => array(
				'rule' => array('validateUrl'),
				'message' => 'valErrDefaultError',
			),
		),
	);
	public function validateUrl($data) {
		$content = array_shift($data);
		try {
 			...
 		} catch (Exception $e) {
			//crazy stuff which is working but NOT RECOMMENDED in this case:
 			//$this->validator()->getField('url')->getRule('validateAvailability')->message
 			//return false;
 			
			//just return the error message you want to replace the default one with
 			return $e->getMessage();
		}
		//no exception thrown - seems to be fine then
		return true;

PS: The commented out code SHOULD NOT be used, but would also work. Those are from my trial and error runs figuring it out. Using the new validator object you can access the properties prior to their usage.

But the simple and correct way here is just to return the error string.
So in our case the form field will have the error message from the exception now.

PS: Returning boolean false will make the original error message defined in your validation array to be displayed, of course.

The book already has quite a thorough documentation on how to Dynamically-change-validation-rules.
Using the validator object basically saves you the trouble working with the $validate array directly.

Avoid "pass by reference" outside of method scopes

<?php
$spell = array("double", "toil", "trouble", "cauldron", "bubble");
foreach ($spell as &$word) {
    $word = ucfirst($word);
}
foreach ($spell as $word) {
    echo $word . "\n";
}

Results in

Double
Toil
Trouble
Cauldron
Cauldron

An issue well known to many (here or here).
So avoid it in view templates where you might re-use the variable somewhere later in the code or always remember to unset it after the foreach loop etc.
The advantage of using small methods in helpers, components, etc is that the scope of the variable is only inside this method and can not affect anything else anywhere.

And in general, trying to avoid the pass by reference is often the best approach. At least where memory is not an issue.

 
4 Comments

Posted in CakePHP

 

More cakephp 2.x infos and 3.x notes

22 May

Behaviors

I just recently found out that the attach() and detach() methods I have always been using seem to be deprecated (or at least only available for BC) – and that one should probably use load() and unload() from now on.
That is sure something I will have to get used to.

Many new features in 2.2

It is worth reading the migration guide on this one. The logging has greatly improved, also the Error Handling, the CakeEmail lib and CakeTime (finally with timezone handling).
A very new feature just recently introduced is the validator object to better interact with the model’s validation rules.

I already started switching some of my apps to 2.2 – although it is still beta it doesnt mean one cannot use it.
It might not be recommended for high profile websites just now. But in most cases it will run at least as stable as 2.1. Only the new features might still be kind of buggy. But since you don’t use them yet you will be fine. But due to a great test case coverage there shouldn’t be too many surprises for your live apps -even if you start using some of the new features.

What I really am looking forward for in 3.0

Namespaces

Of course! Hopefully it will help to remove some of those many collisions we still have in 2.x with class names.

Better approach on aliasing

As outlined in this ticket aliasing as well as correctly merging is broken in the current 2.x version. I fixed it and applied it locally. Unfortunately, the devs are not willing to accept the change as useful fix. I also prosposed bootstrapped aliasing which would make most sense. If declared this early we can avoid conflicts in the controller level.

Better folder naming scheme

As outlined in my proposal @ http://cakephp.lighthouseapp.com they naming scheme for 2.x creates some conflicts – especially in the View folder.
Those should be resolved for 3.x.

Long overdue bug fixes

  • Folder::merge (I applied that fix manually and locally)
  • Containable recursion (also applied manually)

and some minor other ones

Indentation for method comments

Currently they are still not indented. But since the methods are indented (one tab), their comments should be, as well – so both are on the same level. I do that already on the user land site (just as most other php code out there) and hope this will be fixed in 3.0. It is annoying if you want to copy and paste methods from one to another. Lots of indentation correction is necessary.

PSR Framework Standards (or: what I am not so much looking forward for)

Cake2.x already tries to comply with the Psr-0 standard which is about filename conventions in a framework. And the Psr-1 standard is fine, too.
Recently, psr-1 and psr-2 have also been approved as standards. Although many of it reflects the way cake handles it and as it should be there are some major flaws. And I am glad that cake doesn’t follow them (yet).
For starters, the biggest mistake ever in this standard: using spaces for indentation instead of tabs. That’s just so awfully wrong and 19th century that I better don’t even go into any details here. It is sad that even some core developers who tried to reach those guys didn’t succeed in convincing them that they are making a huge mistake.
Secondly the way braces are set for classes and methods (in a new line instead of the same) which stands in contrast to the way it is done for conditional statements where it actually is on the same line. In the end an ugly inconsistency.

The (in my opinion wrong) PSR-1 standard states:

class Foo
{
	public function class bar() 
	{
		if ($var) {
			// ...
		}
	}
}

The correct bracing (which cake fortunately uses) is:

class Foo {
	public function class bar() {
		if ($var) {
			// ...
		}
	}
}

It not only reduces the vertical length (and therefore the readability especially for larger classes and methods) but also keeps the bracing consistent throughout the document.

But not all is bad. There are a few good approaches. And if all those frameworks grow together a little bit due to a common standard it will help the cross-compatibility in the long run.

 
No Comments

Posted in CakePHP

 

Tools Plugin – Part 2: Contact Form

15 Dec

I want to show how easy it is to make a solid and universal contact form in Cake(2).
The full code can be found in the github rep of the Tools plugin.

Model

The most important part first: We need a solid validation for the form. How many forms are out there that do not have any validation. The first thing I always do on about any public contact form: Hit the submit button and smile it if tells me that my empty form "has just been successfully sent". We can do better than that πŸ™‚

class ContactForm extends ToolsAppModel {
	protected $_schema = array(
		'name' => array('type' => 'string' , 'null' => false, 'default' => '', 'length' => '30'),
		'email' => array('type' => 'string' , 'null' => false, 'default' => '', 'length' => '60'),
		'subject' => array('type' => 'string' , 'null' => false, 'default' => '', 'length' => '60'),
		'message' => array('type' => 'text' , 'null' => false, 'default' => ''),
	);
	
	public $useTable = false;
	
	public $validate = array(
		'name' => array(
			'notEmpty' => array(
				'rule' => array('notEmpty'),
				'message' => 'valErrMandatoryField',
				'last' => true
			)
		),
		...
	);

The _schema var mocks a database table so that we don’t really need one. This helps the FormHelper to generate the inputs (maxlength, type, …).
The validation rules will make sure the email is valid and the user actually entered some text.

Controller

The logic takes care of the validation on POST.
/Controller/ContactController.php

<?php
class ContactController extends AppController {
	public $uses = array('Tools.ContactForm');
	public function index() {
		if ($this->request->is('post') || $this->request->is('put')) {
			if (!$this->Session->check('Auth.User.id')) {
				$this->ContactForm->Behaviors->attach('Tools.Captcha');
			}
			$this->ContactForm->set($this->request->data);
			if ($this->ContactForm->validates()) {
				$name = $this->request->data['ContactForm']['name'];
				$email = $this->request->data['ContactForm']['email'];
				$message = $this->request->data['ContactForm']['message'];
	
				// send email with CakeEmail
			} else {
				// error flash message
			}
		}
		$this->helpers = array_merge($this->helpers, array('Tools.Captcha'));
	}
}

As you can see it only sends emails after successfully validating.
For public contact forms I usually like some easy captcha behavior attached so that spam doesnt reach me. You can omit that, of course.
Also note: you should use your own setFlash() method instead of mine!

Last but not least: View

/View/Contact/index.ctp

<?php echo $this->Form->create('ContactForm');?>
	<fieldset>
		<legend><?php echo __('contactLegend');?></legend>
	<?php
		echo $this->Form->input('name');
		echo $this->Form->input('email');
		echo $this->Form->input('subject');
		echo $this->Form->input('message', array('rows'=>15));
		if (!$this->Session->read('Auth.User.id')) {
			echo $this->Captcha->input('ContactForm');
		}
	?>
	</fieldset>
<?php echo $this->Form->submit(__('Submit')); ?>
<?php echo $this->Form->end();?>

Result

Browse to /contact/
That should display the form right away.
After the successful POST you should redirect back to the contact form (emptied then) or to another custom page.

Final notes

Currently, the Model is in the Tools.Plugin. You could put it into your normal app model folder, as well.
But I use it in many projects and therefore I want to keep it dry. Feel free to adjust any of the code to your own needs. Same goes for my own custom methods like $this->Common->flashMessage() etc.

Some might wonder why "ContactController" and not "ContactsController". The latter would be cake conventions. But there are situations where you can and should diverge form those. As in this case where "contacts" would mean more like sth to manage your addressbook. It also makes the url more meaningful out of the box (without any custom routes). With cake2 this all works without any additional customization.

The other thing: I didn’t name the Model "Contact" but "ContactForm" in order to not create possible conflicts which exactly such contacts/contact mangement MVCs (as I did in a contact management suite).

 
11 Comments

Posted in CakePHP

 

Maximum power for your validation rules

07 Oct

I already postet an article about custom validation rules some time ago here.
In this post I introduce some of my custom rules which come in handy quite often.

For being able to use it in different projects I do not use the AppModel for it but a plugin Model class. The AppModel then extends this MyModel class.
I do not want to post the methods here in the blog because they might change over time. You can find the current source code at github.
For Cake2.x the class is: MyModel.

You include it when declaring your AppModel:

App::uses('MyModel', 'Tools.Model');
class AppModel extends MyModel {
}

Don’t forget to CakePlugin::load('Tools') the plugin in your bootstrap.

Lets get started.
The following code fragments are part of the $validate array of the specific model you want to validate.

Validating uniqueness

With this enhanced method we can check on other fields – dependent uniqueness so to speak. It can do a little bit more than the core one.

'field' => array(
	'validate' => array(
		'rule' => array('validateUnique', array('user_id')),
		'message' => 'You already have an entry',
	),
),

In this case the field will only invalidate if this specific user has already an entry.
Two users can have the same field content without interfering with each other.

You can check on as many dependent fields as you want, if the tables are related even across relations:

'field' => array(
	'validate' => array(
		'rule' => array('validateUnique', array('user_id', 'active', 'Role.key')),
		'message' => 'You already have an entry as active member and with this role',
	),
),

Just make sure recursive/contain is correctly set up and it should all work out just fine.

Note: There is now also a contemporary 2.x behavioral approach to this. And the core rule unique is also supposed to be able to check for multiple dependent fields via array syntax. Never tried that, though.

Validating urls

The core rule does not validate "deep" – meaning it cannot check if the url is actually accessible/correct.
My custom rule works like this:

'field' => array(
	'validateUrl' => array(
		'rule' => array('validateUrl', array('autoComplete'=>true)),
		'message' => 'Not a valid url',
	),
),

By default, it will check deep and make sure the url actually exists.
With the autoComplete param you can be more flexible with urls that are missing "http://" etc (note/todo: it would be even better if it would save the autocompleted string). Especially in combination with strict=>true.

If we want to allow only links on the same domain we could say

'field' => array(
	'validateUrl' => array(
		'rule' => array('validateUrl', array('autoComplete'=>true, 'sameDomain'=>true)),
		'message' => 'Please provide a valid url on the same domain',
	),
),

Now /some/link as well as http://samedomain.com/some/link works.
With deep=>false we can disable the deep check for availability.

Validating dates and times

The main improvements are the before/after params:

'end_date' => array(
	'validate' => array(
		'rule' => array('validateDatetime', array('after'=>'start_date')),
		'message' => 'Please provide a valid date later than the start date',
	),
),

There are also date and time only versions like

'time' => array(
	'validate' => array(
		'rule' => array('validateTime', array('allowEmpty'=>true)),
		'message' => 'Please provide a valid time or leave the field empty',
	),
),

I also had to hack around problems regarding empty strings or partially invalid dates. thats why this method is quite long compared to others.

Validating keys (primary/foreign)

'id' => array(
	'validate' => array(
		'rule' => array('validateKey'),
		'message' => 'Invalid primary key',
	),
),
'foreign_id' => array(
	'validate' => array(
		'rule' => array('validateKey', array('allowEmpty'=>true)),
		'message' => 'Invalid foreign key',
	),
),

The id can be either aiid (int10) or uuid (char36), the method will always validate correctly.
Same goes for foreign_id. But due to the allowEmpty it can also be left empty (or 0 for aiids).

Validating enums

Sometimes the enums are "dynamically" generated. You cannot use the build in "inList" validation rule then.
If you have a method set up, you can call this from within the rule:

'field' => array(
	'validate' => array(
		'rule' => array('validateEnum', 'methodX'),
		'message' => 'Invalid value',
	),
),

Somewhere in the model define your custom enum value generator:

function methodX() {
  return array(...);
}

Validating identical

'email_confirm' => array(
	'validate' => array(
		'rule' => array('validateIdentical', 'email'),
		'message' => 'The two fields do not match',
	),
),

As you can see this rule comes in handy if you need to confirm a string to another in the post data. Simply pass the field name along as second param.

Last Words

Hopefully many other cake developers find those enhanced rules useful and maybe they set the core team thinking about implementing some of the features in the future versions of cake.

Please note:
Some of the internal methods, constants etc used might be not available without stuff from my Tools plugin.
Those validation methods above might have to be adjusted to your application. Same goes for the test case which uses some of my own methods/libs to work with.

Update 2013-08

Cake1.x syntax updated to Cake2.x

 
3 Comments

Posted in CakePHP

 

Saving Model Data and Security

21 Sep

In google groups there are quite a few discussions every month about security against (primary key) injection, xss or other things.
And yes, the default templates will not protect you from any of this. They are meant to produce quick "standard" output. But they are not prepared for the real world, at all.
It is actually quite irresponsible not to use custom templates or manually adjust the forms/views.
UPDATE: I could finally convince them to use h() in their default templates! A big WIN for security!

A while ago i wrote about how to create custom templates. See this article for details.
You will also notice, that the controller actions and views are protected against basic – and only basic – injections.
That is mainly xss-attacks. By using h() in the views we can make sure that dangerous strings get destroyed. The custom templates also ensure, that the record we edit/delete exists.

What it still doesn’t do is protecting the model data.

Note: some of the following could be achieved by adding the security component. But sometimes, it is not possible or not desired.

Protection against too many fields

Another tricky one. Again, Firebug can be used to violate your forms. Lets say, a user wants to have more rights – and he happens to know that User.role_id must be 2 instead of 1 in order to be "admin".
In the User edit form (besides "email", "birthday" and stuff) he could then inject a form input called "role_id" which usually is not there.
With a normal $this->save() operation this would actually work! After the next login his role would be 2 (= admin).
What can we do about something like that?

Cake has a "fieldList" as third parameter of the save() function build in:

if ($this->User->save($this->data, true, array('id', 'gender', 'birthday'))) { # note the id (needed here)

Id, gender and birthday are passed, any other field in $this->data will be omitted (especially our role_id^^).
I use this in all sensitive forms (user and role related).

Careful: If you don’t pass the id, it will be omitted and the model cannot set its primary key (and might perform "add" instead of "edit"!

Primary key injection on edit actions

This is the not quite the same as the above topics. All edit actions have the primary key in the forms. I usually remove them, but sometimes I am too lazy to do that (because you need to alter the form url then as well).
So the id actually exists – but won’t be checked prior to the save.
Well, here are some custom solutions – there are more general ones, of course. But I just want to point out the problem and a way to fix it. These could be added to your bake templates, for instance.

Lets take our custom template from above (see link) and modify it:

public function edit($id = null) {
	if (empty($id) || !($data = $this->User->find('first', array('conditions' => array('User.id' => $id)))) {
		//error flash message
		$this->redirect(array('action' => 'index'));
	}
	if (!empty($this->data)) {
		# solution 1: just override with the "correct" one:
		$this->data['User']['id'] = $id;
		# solution 2: check against "correct" one
		if ($this->data['User']['id'] != $id) {
			//BLACKHOLE
		}
		if ($this->User->save($this->data)) {
			//OK
		} else {
			//ERROR
		}
	}
}

Solution 1 is self-explanatory and cannot go wrong.
Solution 2:
After we made sure the record exists, we check the passed id against the record id.
If they don’t match someone tries to trick us. Otherwise the record can be saved.
Solution 3:
Remove the id from the passed arguments (third parameter of save() – but makes it more complicated)
Solution 4:
Put the check in beforeValidate() of your app_model.php. Usually – with the above template you have just "read" the record, so the model has the id in $this->id. It could then return false if the passed id and $this->id do not match. Thats actually my favorite. But I am not sure if it works in all cases. You may only check if you are actually saving a record – meaning: both ids have to be not empty. Only then it makes sense to apply this specific piece of validation.

Protection against missing fields

This is a tricky one. Most beginners (I didn’t either) dont realize that a form can be modified with tools like and Firebug (Firefox Addon) and that Cake would not bother.
Lets say we have a registration form and your gender has to be provided. But you don’t want to. So you simply remove this input from the form and submit it. ViolΓ  – it worked!

Why does it work? Cake automatically applies the validation "needed" for this form. All fields that are passed (it doesn’t matter if they are empty or not – they just have to exist on submit) will be checked. Validation rules for fields that did not get passed will be omitted. And thats our problem. Right here.

OK, we need to make sure that the form fields are actually part of the model data before the validation kicks in.
I wrote a app model extension which can be used – you could also write a model function which one by one adds the required fields if the field does not exist.

Put this in your app model:

/**
 * set + guaranteeFields!
 * extends the core set function (only using data!!!)
 * 2010-03-11 ms
 */
public function set($data, $data2 = null, $requiredFields = array()) {
	if (!empty($requiredFields)) {
		$data = $this->guaranteeFields($requiredFields, $data);
	}
	return parent::set($data, $data2);
}
/**
 * make sure required fields exists - in order to properly validate them
 * @param array: field1, field2 - or field1, Model2.field1 etc
 * @param array: data (optional, otherwise the array with the required fields will be returned)
 * @return array
 * 2010-03-11 ms
 */
public function guaranteeFields($requiredFields, $data = null) {
	$guaranteedFields = array();
	foreach ($requiredFields as $column) {
		if (strpos($column, '.') !== false) {
			list($model, $column) = explode('.', $column, 2);
		} else {
			$model = $this->alias;
		}
		$guaranteedFields[$model][$column] = ''; # now field exists in any case!
	}
	if ($data === null) {
		return $guaranteedFields;
	}
	if (!empty($guaranteedFields)) {
		$data = Set::merge($guaranteedFields, $data);
	}
	return $data;
}

We can now make sure removing any form field will have no effect πŸ™‚ I call it "enforced whitelisting".

Case 1: Using $this->Model->set($this->data) and $this->Model->save() – Important: note the NULL in save(), we may not pass the data a second time!

$this->User->set($this->data, null, array('gender', 'birthday', ...));
$this->User->save();

Case 2: Using directly $this->Model->guaranteeFields($fields, $this->data) and $this->Model->save($this->data)

$data = $this->User->guaranteeFields(array('gender', 'birthday', ...), $this->data);
$this->User->save($data);

Lets combine them

$this->data['User']['id'] = $id; // on edit
$this->User->set($this->data, null, array('gender', 'birthday', ...));
if ($this->User->save(null, true, array('id', 'gender', 'birthday', ...))) { # note the id (needed here)
	//OK
} else {
	//ERROR
}

Now thats protection πŸ™‚
Only the fields that you want to are passed and all the fields that should be there are there. And the primary key is the one it is supposed to be. If your validation rules are correct, you got the perfect form handling, I guess.

Blacklisting

// put this in your app_model.php
public function blacklist($blackList = array()) {
	return array_diff(array_keys($this->schema()), $blackList);
}

Although not recommended for general usage, this might be useful for some rare occasions.
It returns all fields of the current database table except the ones you blacklisted.

Usage:

$this->User->save(null, true, $this->blacklist(array('birthday'));

If you want to pass non-existing fields to the model for validation you need to array_merge() them!

$this->User->save(null, true, array_merge(array('virtual_field'), $this->blacklist(array('birthday')));

Server-side and Client-side Validation

First one is what we just discussed. Client-side usually is JS or browser-specific validation. You should NEVER rely on client-side validation – it can easily be tricked. It is nice to have it – "additionally" in order to prevent unnecessary posts, but the real deal should be what your model validation returns.

SQL Injection

I always smile about other webprojects that are vulnerable to this attack. Its just hilarious. Thats why you use a framework. Because it takes care of the basic problems. And this is one of it.
There are only very few things to be aware of:
Always manually escape manual SQL queries!
CakePHP only takes care of the queries run by $this->save(), $this->saveAll() etc. So if you ever have to write your own SQL script, you will need to do that yourself.

The myth about "required" and "allowEmpty"

They seem to be the same, don’t they? But there is a huge difference between them (the cookbook does mention it here).
"required" is like my "Protection against missing fields", only hard-coded. These fields will validate false, if this field is not present on validation. It does not matter if the field is empty or not. The word "required" only applies to the database field, not its content.
"allowEmpty" is equal to "minLength 1" and ONLY validates this field if the field has been passed to the model. If not, it will not return false. You cannot make sure that a specific field is always filled out with this parameter alone.

If you want to make sure, that the field cannot be avoided AND has to be filled out, you need to combine both rules:

public $validate = array(
	'status' => array(
		'maxLength' => array( # this can be any label you want
			'required' => true, # careful!
			'allowEmpty' => false,
			'rule' => array('maxLength', 5),
			'message' => array('valErrMaxCharacters %s', 5), # a modification of mine to allow better i18n support
			'last' => true # important for more than one rule per field!!!
		),
		... # other rules
	),
	... # other fields
);

The "careful" stands for: this can get ugly really fast. I never use "required" hard-coded. As soon as you use ajax toggling of some fields or forms which only change a small part of the record, you will get validation errors because the required fields are not present (although you don’t even want to change them). You would need to unset() those rules manually. See the above "Protection against missing fields" for a cleaner approach.

Anyway, i hope this clears up things a little bit.

If you really feel that you need to use "required" you should only use it combined with ‘on’ => ‘create’, as well. In 99% of all cases you end up with never validating "edit" validations otherwise. That’t because you often save only a part of the data where you normally wouldn’t provide those required fields.
So as a guideline:

  • Either don’t use required attribute at all and use my whitelisting approach from above
  • Use required with "on create" IF (!) the field always has to be present for all create actions. For other actions use whitelisting and guaranteeFields

A few words to validation rules in general

You can add rules for "imaginary fields", as well. The fields don’t have to exist in the database model. This way you can easily use inputs which are validated separately and combined later on (eg: pwd and pwd_repeat resulting in the final sha1-hashed "password").
Always use "last"=>true for your rules. Unfortunately, the default value is still false here. "true" makes sure that after the first rule already returned false, the following ones are not checked as well (overhead and totally useless).
Also sort your rule array properly. The rules are checked from top to bottom, so it would make sense to put all complicated stuff at the bottom and first check on simple "notEmpty" or "email" rules. The most likely ones to fail at the beginning. Why should you check "isUnique" (database query!) before "notEmpty"? Why should you check an email address exists in the database if it is invalid in the first place?

Last words

Why didn’t I use Security Component for some of the above tasks? Well, it was a bitch to use in the first months. All the time some kind of black holes (white screen of death) without any indication what went wrong. Login/Logout, some other forms as well. With the first ajax forms it got even more messy and i deactivated it at last. Works quite well without it, if you protect the forms manually. With lots of ajax stuff going on you have to do that yourself, anyway…
And against certain attacks the Security Component won’t help anyway (PrimaryKey injection in edit forms etc). So at some point you will have to deal with it yourself. You have to decide to what point. But better sooner than later. If you adjust your bake templates before you start a new project it will save you lots and lots of time while having secure forms and procedures.

Note – 2012-06-02

For Cake2.x please use $this->request->data instead of $this->data in the controller scope.

 
14 Comments

Posted in CakePHP

 

How to implement Captchas properly

09 Aug

What is a captcha?

They protect forms on websites from spammers and bots (@see Wikipedia for details). The main idea: Display some kind of code a human can easily read and submit but a computer can not.

How NOT to implement captchas

This part is even more important, because there is not only one correct way, but even more wrong ways to go here.

Don’t use sessions unless you really have to.
Pages that use something like $_[‘Captcha’][‘field’] and override this one on every form, really freak me out! It makes working with two or more tabs impossible, because they override each others captcha values all the time, resulting in a ****** mess.
You could use an array like structure, but your captcha session array can get pretty big in a short amount of time.

More helpful are captchas which use the current form and some hash values based on the fields + current timestamp. It should not be possible to "guess" or "calculate" the hash value. So there is no way to use future "hashs".
You could use older hashs (like from last week), though. But most bots are programmed to just post right away. They would have to save possible valid "scenarios" for later usage.
Mabye we can come up with something fail-prove later on. For now we want to effectively prevent spam bots to submit their crap without annoying normal users.
The second part is always the hardest. Me – for example – I really hate those image captchas which you can barely read. I often times have to repeat or reload it twice in order to succeed. Annoying comes not even close.

Why do we need captchas

First of all, they make sure that there is no bot (automatic program) posting "spam" or whatever.
But sometimes you just want to add captchas to prevent users from doing some action too often (like friendship requests in a community site etc).

A good example what happens if you dont use captchas, is bakery.cakephp.org.
Some articles have like 36 comments, of which all 36 are SPAM. This is a desaster.
And in this case you even have to be logged in to submit a comment.
The argument that forms for which you need to be logged in don’t need captchas is not contemporary anymore.

Passive Captchas

I already talked about using some hash values based on the fields + current timestamp.
This can be used to generate passive captchas. They are similar to the cake core component "security" which adds some hidden fields to make sure that the fields have not been tempered with.

Both are invisible to the user – they dont even notice the passive captcha. But bots will soon discover that they are facing a wall.
The difference is, that passive captchas should only valid in a specific timeframe. Too fast (less than 2 seconds) is usually a sign for the work of a bot – humans cannot type that fast.
Too late (> x hours?) means you need to revalidate anyway, so we would render the form invalid as well. Well designed forms will keep the posted content, so nothing gets lost.

Another aspect to improve security is to use other user specific fields for the hash value like browser agent (cannot change during posts, but is less secure because it can be modified), IP address (can only be modified by very skilled hackers and therefore is pretty secure), …

Active Captchas

Those are the most commonly used ones. Users either have to read an image, calculate numbers or
interpret a sentence. The first one is not suitable for handicapped people.

Usually they are build as extension on top of passive captchas. We first validate the passive one. If the form is OK, we then validate the user input. If validation passes we render the captcha valid.

I decided to use math captchas. They keep you mentally fit and do what they are supposed to. The only important issue is to make it easy enough. I saw pages using / [division] or numbers above 20 in multiplication or even above 100 for summation – which is total overhead.
But other ones could be used as well – simply by changing configuration settings.

Captcha Behavior

Ok, to sum it up, we want captchas that

  • don’t annoy users
  • protect as good as absolutely necessary
  • can be used with tabs
  • can be easily implemented and configured

The idea is, that we want to add a single form field as well as attach a single behavior to our model.
Thats all there is to it.
That’s why a behaviour in combination with a helper does the trick perfectly.

The code is in my github tools plugin:
captcha behavior
captcha helper
and functionality that both classes use is in captcha lib.

Note: The links are for 1.3 – if you want the 2.0 stuff, you need to switch to this branch.

Setup

First include the plugin in APP/Plugin/Tools and make it available using CakePlugin::load('Tools') or CakePlugin::loadAll().

This makes sure the helper is available in your controller:

public $helpers = array('Tools.Captcha');

Helper usage (in the view):

echo $this->Captcha->input(); // or input('Modelname') if model is different from the form model

Behavior usage (in the controller):

$this->User->Behaviors->attach('Tools.Captcha');
	if ($this->User->save($this->data)) { ... }

Pretty straight forward, isn’t it?

Issues

Current weaknesses (apart from its strenghts):

  • possible "hash extraction" with unlimited use of those valid hashs (session or db to prevent?)

Final notes

Right now it is mainly used for math captchas (active captchas) and just passive captchas.
Feel free to update the missing parts like providing more captcha types (image, sentence, …) or processing types (session, cookie, …).

Since 2.0 the Security component has improved quite a bit. You can combine this captcha behavior with it to get maximum security.
The security component will actually store the form fields as a hash in the DB and therefore invalidate any foul-play right away.

UPDATE 2011-10-13
The i18n translations are now commited, as well: /locale

UPDATE 2012-04-30
For 2.x you need to use $this->request->data instead.

 
29 Comments

Posted in CakePHP

 

Extended core validation rules

19 Jul

I18n Translation

Some translate the rules in the view – but it usually creates redundancy. In some projects this might be intentional. I like to keep the error messages centralized, though.

For that, you can just override the core translation rule – add this to app_model.php:

/**
 * Overrides the Core invalidate function from the Model class
 * with the addition to use internationalization (I18n and L10n)
 * @param string $field Name of the table column
 * @param mixed $value The message or value which should be returned
 * @param bool $translate If translation should be done here
 */
public function invalidate($field, $value = null, $translate = true) {
	if (!is_array($this->validationErrors)) {
		$this->validationErrors = array();
	}
	if (empty($value)) {
		$value = true;
	} else {
		$value = (array )$value;
	}
	//TODO: make more generic?
	if (is_array($value)) {
		$value[0] = $translate?__($value[0], true) : $value[0];
		if (count($value) > 3) { # string %s %s string, trans1, trans2
			$value = sprintf($value[0], $value[1], $value[2], $value[3]);
		} elseif (count($value) > 2) { # string %s %s string, trans1, trans2
			$value = sprintf($value[0], $value[1], $value[2]);
		} elseif (count($value) > 1) { # string %s string, trans1
			$value = sprintf($value[0], $value[1]);
		} else {
			$value = $value[0];
		}
	}
	$this->validationErrors[$field] = $value;
}

Usage (some examples):

var $validate = array(
	'username' => array(
		'notEmpty' => array(
			'rule' => array('notEmpty'),
			'message' => 'valErrEmpty', // some short form to translate
			//normal sentences would of course work, too
		),
	),
	'pwd' => array(
		'between' => array(
			'rule' => array('between', 6, 30),
			'message' => array('valErrBetweenCharacters %s %s', 6, 30), // short form string with variables
			// maybe resulting in something like "Between 6 and 30 chars" as defined in locale.po
		)
	),
	'code' => array(
		'maxLength' => array(
			'rule' => array('maxLength', 5),
			'message' => array('The code cannot be longer than %s chars', 5), // normal text with a variable
		)
	),
	...
);

I like the short forms like "valErrBetweenCharacters %s %s" – they don’t have a certain "grammar", so if you want to change the translation result "Between %s and %s chars" to "Please insert between %s and %s chars" this can be done with one change in locale.po instead of changing all xxx places you used this rule.

Working with dynamic error messages

If you are in need for a way to work with dynamically concatenated message strings you need to use the __construct method:

public function __construct($id = false, $table = null, $ds = null) {
	parent::__construct($id, $table, $ds);
		
	$this->validate['code']['maxLength']['message'] = array('valErrCodeLength %s', Configure::read('Code.maxLength'));
}

This will either override the existing placeholder message or add it to the rule array if it didn’t contain a message param yet.
You can also add complete rules dynamically this way:

$this->validate['code']['extraRule'] = array(...);
// or replace ALL rules for a field:
$this->validate['code'] = array('ruleOne'=>array(...), 'ruleTwo'=>array(...), ...);

Just make sure your rules all contain the last => true setting then.

Custom validation rules

Here is one example how to use a custom validation rule with one param (custom rules can be placed in the model for local validation or the app_model for global validation):

public $validate = array(
	'pwd_repeat' => array(
		'validateIdentical' => array(
			'rule' => array('validateIdentical', 'pwd'), // we want to compare this to "pwd"
			'message' => 'valErrPwdNotMatch',
		),	
	)
);
// in app_model.php:
/**
 * checks if the content of 2 fields are equal
 * Does not check on empty fields! Return TRUE even if both are empty (secure against empty in another rule)!
* //TODO: make it more generic with Model.field syntax
 */
public function validateIdentical($field, $compareWith) {
	return ($this->data[$this->alias][$compareWith] === array_shift($field));
}

Multiple validation rules

The Cookbook states:
"By default CakePHP tries to validate a field using all the validation rules declared for it and returns the error message for the last failing rule"
I don’t really like this default behavior – what sense does it make to validate all if only the last error can be returned anyway. But as long as "last" is not true by default, we have to manually set it:

var $validate = array(
    'login' => array(
        'loginRule-1' => array(
            'rule' => 'alphaNumeric',  
            'message' => '...',
            'last' => true
         ),
        'loginRule-2' => array(
            'rule' => array('minLength', 8),  
            'message' => '...'
        )  
    )
);

The rules are validated top down. So if the first one fails, it will now return the error right away. Usually the following validation rules rely on a positive result of the predecessor. At least, thats how you should arrange your rules.

One example:
[email] (in this order – each with a specific error message)

  • notEmpty
  • email
  • undisposable (custom – per vendor)
  • notBlocked (custom – per webservice)

As you can see, it would not make sense to check all of them every time. It really slows down the validation – especially if the email address is not even valid. So we first want to check the simple stuff and then move on to the advanced (and sometimes more time consuming) rules. All rules get "last"=>true to achieve that.

Other quite handy custom translation rules

They can be put into app_model.php:

/**
 * checks a record, if it is unique - depending on other fields in this table (transfered as array)
 * example in model: 'rule' => array ('uniqueRecord',array('belongs_to_table_id','some_id','user_id')),
 * if all keys (of the array transferred) match a record, return false, otherwise true
 * @param ARRAY other fields
 * TODO: add possibity of deep nested validation (User -> Comment -> CommentCategory: UNIQUE comment_id, Comment.user_id)
 */
public function validateUnique($arguments, $fields = array(), $options = null) {
	$id = (!empty($this->data[$this->alias]['id'])?$this->data[$this->alias]['id'] : 0);
	foreach ($arguments as $key => $value) {
		$fieldName = $key;
		$fieldValue = $value; // equals: $this->data[$this->alias][$fieldName]
	}
	if (empty($fieldName) || empty($fieldValue)) { // return true, if nothing is transfered (check on that first)
		return true;
	}
	$conditions = array($this->alias.'.'.$fieldName => $fieldValue, // Model.field => $this->data['Model']['field']
		$this->alias.'.id !=' => $id, );
	foreach ((array )$fields as $dependingField) {
		if (!empty($this->data[$this->alias][$dependingField])) { // add ONLY if some content is transfered (check on that first!)
			$conditions[$this->alias.'.'.$dependingField] = $this->data[$this->alias][$dependingField];
		} elseif (!empty($this->data['Validation'][$dependingField])) { // add ONLY if some content is transfered (check on that first!
			$conditions[$this->alias.'.'.$dependingField] = $this->data['Validation'][$dependingField];
		} elseif (!empty($id)) {
			# manual query! (only possible on edit)
			$res = $this->find('first', array('fields' => array($this->alias.'.'.$dependingField), 'conditions' => array($this->alias.'.id' => $this->data[$this->alias]['id'])));
			if (!empty($res)) {
				$conditions[$this->alias.'.'.$dependingField] = $res[$this->alias][$dependingField];
			}
		}
	}
	$this->recursive = -1;
	if (count($conditions) > 2) {
		$this->recursive = 0;
	}
	$res = $this->find('first', array('fields' => array($this->alias.'.id'), 'conditions' => $conditions));
	if (!empty($res)) {
		return false;
	}
	return true;
}
// validateUrl and validateUndisposable are to come

More

See the Cookbook for details on that matter (deprecated 1.2).

 
2 Comments

Posted in CakePHP

 

Working with forms

23 Jun

The Cookbook only covers the real basics.
I want to go over some common issues in a more detailed way for Cake1.3 apps.

Dummy Values

Dummy values for select fields are usually added in the view:

/* (1) */
echo $this->Form->input('select_field', array('empty'=>'please select'));
/* (2) */
echo $this->Form->input('select_field', array('empty'=>array(0=>'please select')));

(1) is used if validation is "notEmpty" for this field (key = empty string) or if it is "numeric" and the user has to chose a value other than the dummy one.
(2) is used if validation is "numeric" and you want to allow the dummy value as default value.

Of course, there are other cases, too.

Note: We assume that the controller passes a variable named "selectField" with the options array to the view. You could also manually add options with the ‘options=>array(…)’ param.

Default Values

Many beginners make the mistake to use the options params "value", "selected" or "checked" for the default value. But this field will then be populated with the same default value every time after posting the form. This breaks the logic of most forms. The validation errors are supposed to show what is wrong and the posted content should remain in the form fields.
So how is it done right?

Use the controller for populating the form with default values

/* inside add/edit actions */
if (!empty($this->data)) {
	$this->Country->create();
	if ($this->Country->save($this->data)) {
		...
	} else {
		...
	}
} else {
	/* Now here you can put your default values */
	$this->data['Country']['active'] = 1;
	$this->data['Country']['lat'] = 0.0;
	...
}

It doesn’t matter what type they are (checkbox, select, radio, textarea, text). For checkboxes and radio buttons it should be either 0 (not checked – by default) or 1 (checked).

Default values – HABTM

For HABTM and multiple selects you usually have to duplicate the key since you are using echo $this->Form->input('Country'); (note the capital c to using the model name here) in the form which then becomes sth like <select name="data[Country][Country][]"...> via FormHelper.

if (!empty($this->data)) {
	$this->Country->create();
	if ($this->Country->save($this->data)) {
		...
	} else {
		...
	}
} else {
	/* Now here you can put your default values */
	$this->data['Country']['Country'] = array(1, 3, 9, ...);
}

Remember: You only pass down the keys (usually the primary id) of your records, not the labels/values.

A hot tip if you don’t remember how the default values are passed to the view for more complicated form setups:
Always post your form once and debug $this->request->data in your controller. Take a look at how the array is made up.
Then form your default array exactly this way prior to passing it down to the view and it will work. Neat, isn’t it?

Default Values – hidden!

Many make the mistake to use hidden fields for some values that are not supposed to be edited by users. But they still can be modified using Firebug or can at least be read out (which in most cases is not desired).
So what is the more correct approach here?

if (!empty($this->data)) {
	$this->Post->create();
	// add the content before passing it on to the model
	$this->data['Post']['status'] = '2';
	if ($this->Post->save($this->data)) {
		... 
	}
}

We don’t even create hidden inputs in the form but add the hidden values in the controller "on demand". Right before the model gets them and processes them. No tempering possible then. No overhead in the forms.

2.x Note: Cake2 Security Component does now hash the hidden input content (!), as well. But if you don’t want or can’t use this component the above still applies. And it is usually cleaner than dragging those values unnecessarily through a form again.

Disallowing some fields to be edited

Most beginners would make the (huge) mistake to simply mark the input fields as read-only or hidden field. Well, again, Firebug can easily modify those fields.
If you really want to use this strategy, make sure (!) that they are not passed on to the model. This can be accomplished with the third parameter in the save() method of the model:

// we dont want the "read-only" fields to be edited (like e.g. 'approved')
if ($this->Post->save($this->data, true, array('name', 'title', 'content'))) {

Now only those 3 fields get modified, no matter what. See my security post for details how to secure forms.

A cleaner approach usually is to simply echo the content (without any inputs). For this to work it is important NOT to just pass it along $this->data. After posting the form those fields are not in the data array anymore and will cause errors. The correct approach (I even baked my templates this way) would be to pass the record itself to the view as well:

public function edit($id = null) {
	if (empty($id) || !($post = $this->Post->find('first', array('conditions'=>array('Post.id'=>$id))))) {
		//ERROR and redirect
	}
	if (!empty($this->data)) {
		$this->data['Post']['id'] = $post['Post']['id'];
		if ($this->Post->save($this->data, true, array('id', 'title', 'content', ...))) {
			//OK and redirect
		} else {
			//ERROR + validation errors without redirect
		}
	}
	if (empty($this->data)) {
		$this->data = $post;
	}
	$this->set(compact('post'));
}

Now we have full access to the record even after posting the form: echo $post[‘Post’][‘approved’]; etc.

Validating first (manually)

This comes in handy, if you need to do something prior to saving the posted data:

// very important if you use validates()
$this->Post->set($this->data);
if ($this->Post->validates()) { 
	// do something here
	// false = no need to validate again (already done in validates()!)
	$this->Post->User->save(null, false);
	...
} else { 
	// ERROR 
}

This is also very useful if you don’t want to save the record. Maybe you want to write an email or something.

Note: In save() we do not pass $this->data again because callback functions inside the model could already have altered the passed data. We would override this. So we pass "null" instead.
If you actually need to alter $this->data here, be sure all your important modifications happen inside beforeSave() – as beforeValidate() is now not called anymore with "false" as second param.

Custom controller validations

This is useful if the result of a component matters for the validation process and you want to manually invalidate a field:

$this->Post->set($this->data);
if (!$this->MyComponent->foo()) {
	$this->Post->invalidate('field', 'error message');
}
// if foo() returns false, validates() will never return true
if ($this->Post->validates()) { 
	$this->Post->User->save(null, false);
	...
} else { 
	// ERROR 
}

Modifications prior to validation/saving

Sometimes you want to make sure, that your display field (e.g. "name"/"title") is uppercase for the first letter, and lowercase for the rest. As any other modification as well you would want to put this in the model as beforeValidate() or beforeSave(). Usually the first one is the better choice. After validation there should only be necessary changes that don’t affect the validated fields if possible. And if so, make those don’t break the validation rules. In our case we even wont to auto-prefix urls in order to properly validate them. So here we have no choice other than using beforeValidate():

public function beforeValidate() {
	if (!empty($this->data[$this->alias]['name'])) {
		$this->data[$this->alias]['name'] = ucfirst(mb_strtolower($this->data[$this->alias]['name']));
	}
	/* adding http:// if necessary (if only www... was submitted) */
	if (!empty($this->data[$this->alias]['homepage'])) {
		$this->data[$this->alias]['homepage'] = CustomComponent::autoPrefixUrl($this->data[$this->alias]['homepage']);
	}
	/* very important! */
	return true;
}

If you don’t return true, the saving process will abort. This usually doesn’t make sense in beforeValidate() – and is mainly done in beforeSave().

Note: Most generic beforeSave() and beforeValidate() functionality could easily be transformed into a behavior.

UPDATE 2012-09 – CakePHP 2

For CakePHP 2.x please use $this->request->data instead of $this->data in your controller actions.
Also try to avoid if (!empty($this->data)) {} and use this instead:

if ($this->request->is('post') || $this->request->is('put')) {}

Note that since 2.4 you can also use the array syntax now:

if ($this->request->is(array('post', 'put'))) {}

The rest should be still valid.

UPDATE 2013-03-06 – Custom controller validations

Since 2.x the validation errors are now arrays instead of strings. This also means that the validation does not stop anymore if you used invalidate() as it did in 1.3.
Even with last=>true in your rules it will still trigger the first rule.
You need to use the current master branch of my Tools plugin and the "fixed" invalidate() method as this:

$this->Post->invalidate('field', 'error message', true);

With true as third param you tell it to stop the validation here (similar to last=>true in the validate array) and not to trigger any more validation rules here.

UPDATE 2015-12 – CakePHP 3

With CakePHP 3.x the same still is true: Do not mess in the view with posted data, instead use the passed down entity and/or request data to let the FormHelper output the correct default values.

if ($this->request->is(['post', 'put'])) {
	// Validate/Save
} else {
	$this->request->data['field_name'] = 'defaultValue';
}
 
4 Comments

Posted in CakePHP

 

Validating multiple models at once

20 Jun

There are forms where you want to add/edit not only fields of the current model, but also of a related one (usually hasMany or belongsTo).

Example:
User and Post

$this->Post->set($this->data);
$this->Post->User->set($this->data);
$val1 = $this->Post->validates();
$val2 = $this->Post->User->validates();
if ($val1 && $val2) { 
	// OK (save both models separatly in order to use the user_id)
	$this->Post->User->save(null, false);
	$this->data['Post']['user_id'] = $this->Post->User->id;
	$this->Post->save($this->data, false);
} else { 
	//ERROR 
}

So whats going on here?
We first set the posted data to the models wie want to validate. This is important when using validates() instead of save(). Then we validate them and only if both pass we continue saving.

Why not directly? Why using $val1 and $val2?
Well, for that you have to know something about php as programming language and how conditions are processed.
If (condition1 && condition2 && ...) {} else {} stops checking conditions as soon as the first condition fails. this is "smart" because it saves time. If the first condition fails, it will jump to the else block no matter what the other conditions return. So why bothering to check them?
But we want to validate both models no matter what – so we have to process them before and only compare the return values! If you miss that you won’t see the validation errors of the second model in the view (if the first one failed, anyway).

Cake automatically passes the errors to the view, if the Model.field syntax is used there:

echo $this->Form->input('User.email');
echo $this->Form->input('User.username');
echo $this->Form->input('comment'); // or Post.comment

Thats it. You can extend this with as many models as you like.

Using PHP Tricks

If you read my other article you might have found another approach:

if ($this->Post->validates() & $this->Post->User->validates()) {}

This works because one & will first check all conditions before deciding what to do.

To summarize: NEVER use two & (&&) in this situation – for validating multiple models at once. It’s wrong!

Using cake’s find(all)

If your data is in model + related models structure you can also use this approach. It is faster and shorter (but I sometimes like control over my form processing):

if ($this->Post->saveAll($data, array('validate' => 'only'))) {
	//saveAll with validate false or single set of saves with validate false
}

This will only validate all the inputs without saving it.
Either way you should then set validate to false afterwards (no need to re-validate the data on save).

 
1 Comment

Posted in CakePHP