screenshot-2016-09-29-09-44-44

Laravel Envoyer & FreeBSD mv: illegal option — T

I have been meaning to try Laravel Envoyer for some time now. At work we have a deployment process which is mainly focused around Codeship with some custom scripts that are either on the server or committed to the repo.

Although there are many options for zero downtime deployment I really like the idea of using Envoyer so I decided to give it a go. Nearly all of our servers are FreeBSD for various reasons and as I excitedly watched the deployment happen for the first time I was disappointed when I saw ‘Finished With Errors’. I clicked on the output button with provides you with some more feedback and I was faced with a rather confusing message.

mv: illegal option -- T
usage: mv [-f | -i | -n] [-hv] source target
mv [-f | -i | -n] [-v] source ... directory

screenshot-2016-09-29-09-10-11

It looks like the option -T is not available on FreeBSD as it is in Linux systems so the final step where Envoyer activates the new release is not able to move the ‘current-temp’ folder to be the ‘current’ folder.

mv -Tf /home/forge/domain.com/current-temp /home/forge/domain.com/current

I sent a tweet out to see if anyone else was having this issue or if it was something I was doing wrong. I got a helpful tweet back from @FreeBSDhelp which confirmed that the option -T was not available.

Interestingly if you run the move command without the -T and you already have a current folder it will place the ‘current-temp’ folder inside the ‘current’ folder resulting in ‘/home/forge/domain.com/current/current-temp/’, obviously not what we are looking for. So it looks like the option in FreeBSD is to replace the -T option with -h which will do a similar thing.

So for FreeBSD we need to run

mv -hf /home/forge/domain.com/current-temp /home/forge/domain.com/current

and on all other we can run

mv -Tf /home/forge/domain.com/current-temp /home/forge/domain.com/current

I have exchanged a couple of emails with Taylor and I hope that we will see this merged into Envoyer soon so that I can continue my trial of the product.

I would love to hear from anyone else who has had the same issue.

laracon-us
Video

Laravel 5.3 Overview

I am sure many of you who follow Laravel will know that it was Laracon US just a few weeks ago (July 27-29, 2016 in Louisville, Kentucky). I purchased a live streaming ticket because I was unable to attend the conference this year.

The website that is hosting the video has just released all the videos for free open access now, so get yourself along to https://streamacon.com/video/laracon-us and enjoy!

It’s also worth keeping your eye on Laracon EU which is when it’s rumoured Laravel 5.3 will be released!

Screenshot 2016-08-16 08.53.52

 

Some of my other favourites include:

Source: Watch Taylor Otwell – Laravel 5.3 Overview from Laracon US!

Screenshot 2016-07-03 13.30.51
Link

Laracogs

Prepare your Laravel apps incredibly fast, with various commands, services, facades and boilerplates.

I stumbled across Laracogs a few days back. I setup a test project to play around with it and I found both the CRUD and the Table CRUD to be really interesting.

I recently chatted with Matt Lantz, from Yab Inc, who created the package. Chatting in a PM in the wonderful LaraChat slack group, he informed me he is working on a version 2 of the package. This is really exciting because it looks like you will be able to pick from a few DDD (Domain Driven Design) patterns.

https://laracogs.com

photo-1416862291207-4ca732144d83
Link

The 10 Buffer Values

I absolutely love Buffer as a company and I find their culture and values fascinating. If you have not already stumbled across their blog I would highly recommend you head over to open.buffer.com. Below is a direct link to the Buffer Values, enjoy!

The 10 Buffer Values and How We Act on Them Every Day

Screenshot 2016-06-27 10.55.52

Get remote laravel log file

By now you are deploying your web applications via some sort of magic like Codeship and have probably decided to turn FTP off on your servers. Either that or you are using Forge and the only way you can get at your production server is via SSH.

If you want to run migrations on your production server then you can SSH in or maybe have an Envoy command to do this. However, what happens if you need to review your production log files from your Laravel app but you cannot just FTP in and download them.

At first I thought about doing it via Envoy but I could not get my head around how to use the ‘scp’ command. I ended up setting an alias to do it from my local machine using

alias get-amazing-com-log=”scp username@server.com:/home/web/amazing.com/app/storage/logs/laravel.log ~/Projects/amazing_com/app/storage/logs/”

Then I thought, how about a quick Laravel Command to do something like

php artisan logs:get

To do this we are going to make use of the SSH Class in Laravel, specifically SFTP Downloads. For us to be able to use this class we need to set some remote configurations.

To get started we need to go to ‘/app/config/remote.php’ and add the settings for your remote server, in this example we will set them for our production server.

<?php
// app/config/remote/php
'connections' => array(
    'production' => array(
        'host' => $_ENV['SSH_HOST'],
        'username' => $_ENV['SSH_USERNAME'],
        'password' => $_ENV['SSH_PASSWORD'],
        'key' => '',
        'keyphrase' => '',
        'root' => '/home/web/amazing.com',
    ),
),

You will notice that the only thing we actually set in this file is the ‘root’ as everything else is extremely sensitive so we are going to set those configuration items using “dot” files.

<? // .env.local.php return array( 'SSH_HOST' => 'server.com',
'SSH_USERNAME' => 'deployment',
'SSH_PASSWORD' => '53curep455w0rd',
);

Once we have the configuration setup then we are ready to setup our new Artisan Command.

<?php
// app/commands/GetLog.php

use IlluminateConsoleCommand;
use SymfonyComponentConsoleInputInputArgument;

class GetLog extends Command
{

    protected $name = 'logs:get';
    protected $description = 'Connect via SSH to the given remote environment and download log file.';

    public function __construct()
    {
        parent::__construct();
    }

    public function fire()
    {

        $this->info('Stand by...');

        try {

            $env = $this->argument('env');
            $app_log_path = '/app/storage/logs/laravel.log';
            $remote_log_path = Config::get('remote.connections.' . $env . '.root') . $app_log_path;
            $local_log_path = dirname(__FILE__) . '/../storage/logs/' . $env . '.log';

            SSH::into($env)->get($app_log_path, $local_log_path);

        } catch (InvalidArgumentException $e) {

            $this->error('Check you have settings for app/config/remote.php connections.' . $env);
            exit;

        }

        $this->info('Environment : ' . $env);
        $this->info('Connected to ' . $_ENV['SSH_HOST']);
        $this->info('Getting file from : ' . $remote_log_path);
        $this->info('File downloaded to : ' . $local_log_path);

    }

    protected function getArguments()
    {
        return array(
            array(
                'env',
                InputArgument::OPTIONAL,
                'Environment to get the logs from set in app/config/remote.',
                'production'
            ),
        );
    }
}

And remember to register this command so you have access to it when running Artisan on the CLI.

<?php

// app/start/artisan.php

Artisan::add(new GetLog);

And there you have it. Now you can run a simple Artisan command from your local machine or from within your Homestead VM and it will connect to production, pull down the log file and store it in ‘app/storage.logs/production.log’.

You’re welcome.

Clicksco PHP Flow

This is the general flow in the PHP family at Clicksco. As of June 2014.

As an international company the general business relies a lot on email, Basecamp and Skype however as a tech team we have an arsenal of different tools and services we use and I would like to share our processes with you.

Instant Messaging and Beyond

We used Campfire for a long time; however more recently have moved to HipChat. Everything, and I mean everything, is centered around HipChat. We have rooms for general discussion, Design, Frontend, internal frameworks, PHP, .NET, Laravel, Music, Infrastructure Management, Project Management and the most important rooms of them all, project rooms.

Like GitHub, we like to centre everything possible into project rooms and have Clickbot (our version of Hubot) running in all rooms. Although we do not rely on Clickbot as much as GitHub relies on Hubot, we do rely heavily on the ethos that if it is not in a project room then it is not happening! Most of the team have the HipChat app on their mobile so if they are away from their desk and you @mention them in a room they will get a notification. It is then up to them how and when they respond.

Source Control & Project Management

As a company we use BitBucket as our central remote repository solution; however in the PHP family we have our primary remote repository (origin) set to a Codebase repository. Codebase classes itself as “Professional code hosting, with a complete suite of project management tools built-in.” Every project we work on has a Codebase project and that project may have one or many repositories. The most used feature in Codebase is the ticketing system, which we use to assign features, enhancements and bugs to members of the family. We then set ticket priorities and can give them relevant tags. Each Codebase project is then linked to a HipChat project room so that we get notifications on commits, ticket creation and ticket updates.

Agile Software Development

We follow the basic principles of SCRUM and so our day starts with a standup outlining what we worked on yesterday, what we have planned for the coming day and highlighting anything that is blocking progress. The standups are quick and done via Google Hangouts (because HipChat does not yet support team video chat).

Communication

Throughout the day HipChat is always open and is our central communication tool throughout the entire tech team, not just the PHP team. With the recent launch of individual video calling in HipChat we frequently have one to one calls to discuss ideas, issues and problems. HipChat also allows us to share our desktop screen with the other caller. Although we do use HipChat for this, we also frequently use Screenhero which has the added feature of allowing live collaboration. It is sometimes much easier to just let the other person click and type. Usually a request for a video call or screen share will be done in a project room in HipChat so itis nice to have the new ‘/hero @jamesmills’ command for HipChat which will open a Screenhero session with the user specified.

Source Control Workflow

All the tech teams at Clicksco use Git and in the PHP family we follow GitHub Flow. We all work on branches which are synced remotely on each commit. Once we are happy with a new feature or a bug fix, we create a merge request in Codebase and set one of our fellow family members to be the person responsible for the merge. This brings us nicely onto one of the most important steps in a projects lifecycle: Code Reviews. Every line of code committed will get reviewed by at least two developers. Code reviews are a fundamental part of our process. A code review helps to bring our team together, widen knowledge, pull everyone to the same level, give a sense of ownership and achievement, catch bugs, discover new feature enhancements and possible refactor opportunities.

Deployment

Deployment is always an interesting subject and I still find it staggering that in 2014 people are using FTP to download/upload files manually. For the situations where we have to use FTP to deploy to server we have a DeployHQ account (by those nice people at Atech Media). Our main deployment tool at the moment is Codeship. Codeship is proving to be an amazing tool and I would recommend you to try out their service, they have a free plan.

Once something is merged into the master branch, Codeship takes over. Codeship will run tests on all branches. If the merge is made on the master branch and all the tests pass, then Codeship will then run custom deploy scripts. As we have a number of projects deploying to a number of servers we keep all our deployment scripts in source control.

Obviously, whichever tool is being used, everything is set to report back to the project room in HipChat. You can literally see the commit, merge, test status and deploy result just by sitting in the HipChat room.

Bug Tracking

Once a system is running live on production we make use a service called BugSnag which reports any errors and exceptions on a given environment. For example, if an exception is thrown, then BugSnag will pick this up, automatically create a ticket in Codebase and notify the project room in HipChat. A member of the team will jump on the ticket, update its status, create a branch, commit code, merge request the branch and code review will take place. The merge will get approved, deploy will kick in, ticket will automatically get updated with ticket reference in the commit message and every single step will get logged via notifications to the project room in HipChat.

Communication tools and keeping up-to-date

We also have rooms in HipChat to discuss things like Laravel. Having things like new Laravel.io podcasts and new lessions on Laracats all fed into a central location is all part of the ethos so we use Zapier to monitor RSS feeds and post notifications into the rooms when something new is released. We are working on getting Clickbot to help us with this.

Is this the only way?

No, not by any means. I wanted to write this blog post to share what we do in the PHP team at Clicksco, at the moment. One of the main reasons I added the month and year at the start of this post is because there is no way of knowing when it will be outdated, probably as soon as next month. We have got to this stage by having a great team who are always bring new tools and services to the table and are willing to try new things. The key thing is that there is a method, there are principles and a procedure. Everything and everyone is open to change, but if you do not have an established process in the first place, you have no process to improve.

We also have a site that documents our flow and our tools that is available to view at http://php.clicksco.com. Please let me know your thoughts in the blog comments.

Conclusion

So that is about it. A brief summary of what we use and how we use it. As I am sure you have established, there is one tool that is a hub to all our activities. Having HipChat, although anything like Campfire or Slack would work, as a central hub has proven fundamental to improving communications and efficiency in the PHP family at Clicksco.

Why the emphasis on a central notifications hub like HipChat? Anyone, at any time, from any location, can go into any project room and know exactly what is happening. You can scroll through the room history to quickly get up to date and you can see live updates from everyone and every tool we use all in one place.

Soon we will be able to use Clickbot to issue more commands from within HipChat, heck we might be able to just all work from within HipChat.

Limit packages to development in Laravel applications

Let’s say you like to use Laravel 4 Generators by Jeffery Way like we do at Clicksco. If you are not familiar with this package then you should head over to Laracasts and watch the free video. You will only be using the generators in development so you don’t really want to have this included on production. Let’s have a look at how we can do this.

First of all you need to make sure that you have included ‘way/generators’ in the ‘require-dev’ section of your composer.json file and not the ‘require’ section. You can do this manually or from terminal by running

php composer require --dev "way/generators:2.*"

Worth noting that if you run this then composer will automatically run a ‘composer update’ and this update will run including require-dev packages, this is the default action of a composer update. Your composer.json file will now include the package in the ‘require-dev’ section.

...
"require-dev": {
"way/generators": "2.*"
}
...

After requiring the package, which now sits in your vendor folder, you will need to instruct Laravel to use it. To do this we register a service provider. If you were to follow the installation instructions on GitHub then you will notice that you are asked to open ‘app/config/app.php’, and add a new item to the providers array (shown below).

...
'providers' => array(
'WayGeneratorsGeneratorsServiceProvider',
),
...

This is where we are going to start doing things a little differently.

Instead of doing the above we are actually going to add the service provider to our development environment app config.

As I am sure you are aware, inside the ‘app/config’ folder you can create folders that match your environments. By default the ‘app/config/app.php file will be loaded by Laravel and this will be true for production. However, for development we want to tell Laravel to load ‘app/config/development/app.php’. To do this we need to alter ‘bootstrap/start.php’ and include something like:

<?php
...
$env = $app->detectEnvironment([
'development' => [
'Jamess-MacBook-Pro.local',
'precise32',
'localhost.dev',
'localhost',
],
'production' => [
'live.mydomain.com'
]
]);
...

The above example has my local machine hostname for situations where I would be using MAMP and also the hostname of my Vagrant VM for when I am working in our default Vagrant setup we have at Clicksco. I then have the domain of the live application in the production array. There are a number of different ways of managing this but I have found this to work best for my needs.

Another way, and one that is much better

<?php
...
$env = $app->detectEnvironment(function () {
return isset($_SERVER['ENV']) ? $_SERVER['ENV'] : 'development';
});
...

We have told Laravel to check in the ‘app/config/development’ folder for settings to be used when in our development environment we now need to add our service provider to this file.

Normally we would just add the config settings for development in a new array inside the ‘app/config/development/app.php’ file but by default this will override the settings loaded from ‘app/config/app.php’. So we are going to make use of the ‘append_config’ helper method in our environment app configuration file, see below.

<?php

return array(

'debug' => true,

'providers' => append_config(
array(
'WayGeneratorsGeneratorsServiceProvider',
)
),

);

Notice that the debug = true is an example of an override and our providers = append_config is an example of us appending to what has already been loaded. Now we have access to Jeffery Ways Generators but only when we are in development, you can test this by running

php artisan

and you will now see something like

[text]
generate
generate:controller Generate a controller
generate:migration Generate a new migration
generate:model Generate a model
generate:pivot Generate a pivot table
generate:publish-templates Copy generator templates for user modification
generate:resource Generate a new resource
generate:scaffold Scaffold a new resource (with boilerplate)
generate:seed Generate a database table seeder
generate:view Generate a view
[/text]

To finish off we will want to make sure that we don’t include the ‘require-dev’ packages on production. This is simply done by running composer update adding the –no-dev option. Just add the ‘–no-dev’ option to your deployment script.

composer update --no-dev

Hope this helps someone.

Clicksco are looking for awesome people, are you one of them?

Dubai-based Clicksco employs more than 200 people in nine offices and six countries. It has a turnover of some US$100 million and serves more than 11 million customers per month through its network of comparison shopping, B2B, ecommerce and travel websites.

The technical arm of Clicksco is based in Middlesbrough in the UK and we are looking for new people to fill the below opportunities:

  • Product / Project Management
  • Front End Development / Inc Word Press
  • .net c#
  • Design
  • iOS
  • Android
  • PHP

If you are interested then please contact us@thap.co.uk