RSS
 

Posts Tagged ‘GitHub’

CakePHP and connecting to GitHub API

22 Oct

I had to write a small tool to automate releasing certain GitHub repositories, and for that to authenticate I had to connect to GitHub API.
The integration wasn’t super-easy as there was no documentation yet on how this could be done. But I finally figured it out and want to share it.

HybridAuth plugin

I first introduce the plugin pretty quick I chose to use as authentication piece.
HybridAuth is maintained by a CakePHP core developer and bridges the original HybridAuth implementation into CakePHP. That library aims to "act as an abstract API between your application and various social APIs and identities providers". Out of the box it provides quite a few very popular services to connect to.

Getting started

I did install the plugin as documented, I also made sure the Migration file for it has been included because
we do need a "social_profiles" table here.

Then I connected the Users and SocialProfiles table:

    /**
     * @param array $config The configuration for the Table.
     *
     * @return void
     */
    public function initialize(array $config) {
        parent::initialize($config);
        ...
        $this->hasMany('ADmad/HybridAuth.SocialProfiles');
        EventManager::instance()->on('HybridAuth.newUser', [$this, 'createUser']);
    }
    /**
     * @param \Cake\Event\Event $event
     *
     * @throws \RuntimeException
     *
     * @return \App\Model\Entity\User
     */
    public function createUser(Event $event) {
        // Entity representing record in social_profiles table
        $profile = $event->data()['profile'];
        $username = substr($profile->profile_url, strrpos($profile->profile_url, '/') + 1);
        $user = $this->newEntity(
            [
                'username' => $username,
                'email' => $profile->email
            ]
        );
        $result = $this->save($user);
        if (!$result) {
            throw new \RuntimeException('Unable to save new user:' . print_r($user->errors(), true));
        }
        return $result;
    }

I used the "profile_url" data to automatically generate the same user on my website.
Since the login was only allowed via GitHub login, there was no change of collision.

Then I made sure the HybridAuth authentication adapter is added to the list of components in the AppController:

    /*
     * @return \Cake\Network\Response|null|void
     */
    public function initialize() {
        parent::initialize();
        ...
        $this->loadComponent('TinyAuth.Auth', [
            'authenticate' => [
                'Form',
                'ADmad/HybridAuth.HybridAuth',
            ],
        ]);
    }

I also modified the login according to the documentation.

And finally I just needed a link in the navigation menu in the case the user is not logged in yet:

echo $this->Html->link(
    'Login with GitHub',
    ['plugin' => false, 'prefix' => false, 'controller' => 'Account', 'action' => 'login', 
        '?' => ['provider' => 'Github', 'redirect' => $this->request->query('redirect')]
    ]
);

Note that the "redirect" query string is only necessary for CakePHP 3.4+ when the session is not used anymore for remembering the location to redirect to after login. And also note that at this point only a "dev" branch of the plugin supports the 3.4+ version yet.

Figuring out the configuration

Now that was the most difficult part. With a lot of debugging I found out that since the GitHub provider is not one of the core ones I need to provider wrapper path and class here:

    'HybridAuth' => [
        'providers' => [
            'Github' => [
                'enabled' => true,
                'keys' => [
                    'id' => env('AUTH_ID_GITHUB', ''),
                    'secret' => env('AUTH_SECRET_GITHUB', '')
                ],
                'wrapper' => [
                    'path' => ROOT . '/vendor/hybridauth/hybridauth/additional-providers/hybridauth-github/Providers/GitHub.php',
                    'class' => 'Hybrid_Providers_GitHub'
                ],
                'scope' => 'user:email,repo'
            ]
        ],
        'debug_mode' => false,
        'debug_file' => LOGS . 'hybridauth.log',
    ],

Note that I also set custom "scope" permissions here, you can however here leave that out or add more.

Small tweaks

I didn’t want to use the the plugin controller action to authenticate, but my own in AccountController (in order to execute a few custom things upon login).
So I just overwrote the hauth_return_toURL:

'hauth_return_to' => [
	'controller' => 'Account', 'action' => 'authenticated', 'plugin' => false, 'prefix' => false
]

Testing

Yeah, ok, here I did cheat.
The hybridauth library has a little flaw that makes it difficult to connect to CakePHP as plugin: It always forces the session to be started right away. Especially when testing the controllers now this can be super annoying as it throws ugly warnings:

..Warning Error: session_start(): Cannot send session cookie - headers already sent by 
(output started at phar:///home/vagrant/Apps/.../phpunit.phar/phpunit/Util/Printer.php:134) 
in [/home/vagrant/Apps/.../vendor/hybridauth/hybridauth/hybridauth/Hybrid/Storage.php, line 20]

So I just added the adapter when not in CLI mode:

if (PHP_SAPI !== 'cli') {
	$authenticate['ADmad/HybridAuth.HybridAuth'] = [
		...
	];
}
$this->Auth->config('authenticate', $authenticate);

Tests are green again 🙂

In short

All in all HybridAuth is a great CakePHP plugin to connect this HybridAuth library and any OpenID and OAuth authenticated service to your application.
Give it a spin!

Besides the here mentioned GitHub provider I also managed to use Facebook/Google sign-in this way in another app.
Basically all apps, if for technical users or more a social network, can benefit from such a one-click login as it really takes away the pain of double-opt-in registration forms and alike.

 
No Comments

Posted in CakePHP

 

Continuous Integration with Jenkins

04 Mar

CI with Jenkins and GitHub is especially interesting for private repositories, as CI with Travis is mainly for free GitHub repos (unless you have Travis Pro, of course).
But since Jenkins is OpenSource and free, it might make sense to set up this on your server and run CI directly in-house.
It integrates nicely with GitHub. So the following tutorial will focus on that example.

So for the beginning we have our example app in GitHub, e.g. a CakePHP app (optionally including composer dependencies).

The main goals are:

  • Continuous test results on each push (especially to master).
  • Automatic test results for each PR (very important when working in teams) – linked inside the PR just as Travis would.
  • Coverage being analysed to see where the weak points can be (classes with < 50% test case coverage).

Set Up Jenkins

This is probably the most difficult part of all.

We first set up Jenkins.

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | apt-key add -
nano /etc/apt/sources.list
------------------------------
deb http://pkg.jenkins-ci.org/debian binary/
------------------------------
apt-get update
apt-get install jenkins

Check if Jenkins runs properly

http://hostname.domain.local:8080

Install plugins and all their dependencies

Let’s then include the plugins we need to integrate GitHub: "GitHub Pull Request Builder" and "GitHub plugin".
The first is especially interesting as it makes it possible to have Jenkins "build/pass" statuses directly in each PR (similar to Travis).

Access Data -> Global Access Data -> Add Access Data

Create access data for the repository you want to access. Provide username and password or keyfiles.

Manage Jenkins -> Configure System

  • Add your repositories under "Git"
  • Set "Manually manage hook URLs" under "GitHub Web Hook"
  • Generate token (https://github.com/settings/applications) with Github user which has access to the
    private repository and put the token under "GitHub pull requests builder"

Add Job

  1. Select "Free Style" project and provide a jenkins working space name
  2. Put your repository url (e.g. https://github.com/Name/repo) to "GitHub-Project" input field
  3. Select Git under "Source-Code-Management"
    Put your repository url again and select the access data from step 5 of this tutorial.
    Click on advanced settings and set "+refs/pull/:refs/remotes/origin/pr/" for Refspec and "origin" for name.
    Set also the branch specifier to "${sha1}"
  4. Select "Build when a change is pushed to GitHub" and "GitHub pull requests builder" under "Build trigger"
    Add persons or organizations to "GitHub pull requests builder" which are whitelisted for trigger an PR build request
  5. Add shell execution script for your test cases under build proceed (see below)
  6. Add "Set build status on GitHub commit" as post build action.

My shell execution script for the CakePHP apps looks like this:

cd /var/lib/jenkins/jobs/projectname/workspace/app
cp /var/www/projectname/app/Config/database.php Config/
mkdir -p tmp && chmod -R 777 tmp && rm -R tmp/*
cp /var/www/projectname/app/composer.phar ./ && php composer.phar update
Console/cake test app AllApp --stderr

It first navigates into the freshly pulled repo code and into the APP dir.
Then it copies over the database.php from the current staging website (could also be some other place where you store them).
It then creates tmp dirs and sets the correct permissions.
The composer.phar file is also copied over and executed to pull all composer dependencies.
At finally the AllApp (group) tests are executed – with –stderr in order for the session to work properly.
One can then also apply -q for silent (=none) output.

Setup GitHub hooks

To trigger the build process on Jenkins, we need to configure service hook on GitHub. In order to achieve this, navigate to GitHub repository settings and configure Jenkins Hook URL for GitHub plugin. The URL format http://<jenkins-username>:<jenkins-password>@<Elastic-IP-Address>:8080/github-webhook/

Running tests

Run first test manually

Click on "Build with Parameters" and hit "Build" and you will see the first manually triggered build in progress.
If all goes well the icon should be blue (pass) – not red (fail).
In case you need to see what is going on, check the "Console Output" of a specific build.

Check automated tests

Commit to master and see if a build is triggered. Also open a PR and check if there is a badge displayed with the current build status.

Sugar

Badge

To have a small "build status: passing" badge in our Readme, we can install the plugin "Embeddable Build Status".
It will create a new menu entry from where you can select your badges including ready-to-use Markdown syntax if needed.

And if it is a private repo you might also want to htaccess the Jenkins web interface or protect it some other way from the public.

Code Coverage

You can install the plugin "Clover PHP Plugin".
Create a folder "coverage" in your workspace.

Add this to the above command:

--log-junit ../coverage/unitreport.xml --coverage-html ../coverage --coverage-clover ../coverage/coverage.xml

And set up a "Publish Clover PHP Coverage Report" Post Build Action with the absolute path to this coverage.xml file in the workspace.
In my case just coverage/coverage.xml as it can be relative to the workspace, as well.
If you also want to publish the HTML coverage report (which is pretty neat), you can simply put coverage in there then.

Make sure you got Xdebug up and running (apt-get install php5-xdebug). It is a direct dependency here.
Note that using code coverage reports will slow the whole process down by "x3" at least. Usually that shouldn’t really make a difference, though.

In most cases you want to exclude few dirs. In case you are already using a phpunit.xml file which resides in your APP dir, you can easily add

<phpunit>
	<filter>
		<whitelist>
			<directory suffix=".php">*/Controller</directory>
			...
		</whitelist>
		<blacklist>
			...
		</blacklist>
	</filter>
</phpunit>

You can also just create this file and also copy it over to the jenkins workspace.

Memory Limit

If you have a project of medium to large size you will soon see the memory usage for the test coverage of 50%+ way above 200MB.
With more coverage and the project getter bigger, or some coverage report generated you will run out of memory with 256MB (which one would think should suffice).
Raise your limit to at least 512 for the PHPUnit testing. You can use the phpunit.xml file:

<phpunit>
	<php>
		<ini name="memory_limit" value="512M" />
	</php>
	...
</phpunit>

Cleanup

Use the plugin "Discard Old Build plugin" to keep your diskspace reasonable. Without it (especially with CodeCoverage) the disk space
will soon exceed several GB. We usually keep the last 40 builds.

More

This article goes more into detail about the installation of Jenkins and also how to use additional tools like CodeSniffer, MessDetector and CO using a build file and Ant. But I didn’t check that out yet.
There is also jenkins-php.org if you want to go all in 🙂

 
No Comments

Posted in Testing