How to Use a PHP Closure to Implement a SoapClient Method Staggering Retry

The Problem

I ran into an situation where I have to interact with an extremely unreliable SOAP web service using PHP’s SoapClient. I needed a way to consistently retry failures before eventually dying if it doesn’t ever succeed.

I prototyped this using procedural code, but when I implemented it for real, it was in object-oriented fashion. Procedural is an easier way for me to prototype and to explain programming ideas.

Imagine you have a Soap webservice called butt that poops.

Normally, you would do something like this.

try {
    $butt = new SoapClient();
    $butt->poop();
} catch (SoapFault $sf) {
    echo $sf->getMessage();
}

But now imagine that your butt doesn’t work right all of the time.

The Solution

To deal with this irregularity, let’s automatically retry poop to see if it eventually works.

In this example, I’m going to create a function called retriable.

/**
 * This function attempts to call $method on $client the $numRetries 
 * specified waiting for the $wait in seconds for the $maxWait 
 * in seconds.
 * 
 * If $wait is -1, rand(1, $maxWait) is used to determine wait time. 
 * This is meant to juggle requests that all may have come in at 
 * the exact same time. All of these requests hammering the
 * SoapServer at the exact time is causing performance degradation. 
 * By juggling the $wait, it could help spread out the requests.
 *
 * @param SoapClient $client
 * @param string $method Name of SOAP method to call
 * @param array $params An array of parameters to pass to the SOAP method, 
 *   if not use array()
 * @param integer $numRetries The number of times to retry
 * @param integer $wait The number of seconds to wait, if -1, retries 
 *   will wait between 1 and $maxWait seconds.
 * @param integer $maxWait The maximum number of seconds to ever wait.
 */
function retriable(SoapClient $client, $method, $params, $numRetries, $wait = -1, $maxWait = 4)
{
    $retry = 1;
    $waitInSeconds = function() use($wait, $maxWait, $params) {
        $ret = 1;
        if ($wait != -1) {
            $ret = rand(1, $maxWait);
        } else {
            $ret = $wait;
        }
        echo $ret . "\n";
        return $ret;
    };

    while ($retry <= $numRetries) {
        try {
            return call_user_func_array(array($client, $method), $params);
        } catch (\SoapFault $sf) {
            // If num retries is hit, throw the SoapFault
            if ($retry == $numRetries) {
                throw $sf;
            }
            $retry++;
            sleep($waitInSeconds());
        }

    };
}

Now instead of my original implementation, to call the SOAP method, I do it like this.

try {
    $client = new SoapClient();
    retriable($client, 'poop', array(), 3, 9);
} catch (SoapFault $sf) {
    echo $sf->getMessage();
}

This will call $client->poop().

Where’s the Closure?

The closure came into to play when calculating the number of seconds to wait. Obviously, I could have just used the $wait parameter value. But in my situation, the problem with the SOAP web service was that if a bunch of requests came through at the same time, the backend system would shit itself.

That happened a lot in my case because the requests coming into my application were from another business system sending me SOAP requests. Sometimes in the middle of the night, it would dump a bunch of requests on me all at once. While my code had no problem handling the requests, when I interacted with this other Soap web service, it struggled to keep up.

That’s why I concocted this retry method, with the additional feature to not just retry on a fixed number of seconds. If they all retried in 4 seconds, the backend system would continue to choke on the requests because they’d all come through again at the same time. By using PHP’s rand function, I could put some distance in time between retry attempts.

While this method will never totally alleviate the issues with this poorly performing business system, it did mitigate a lot of exceptions that required a human to get involved.

How-To: Git Merge

I create a lot of Git branches. Every change I do is a branch. That means I perform a lot of Git merges. Here’s how I do it.

Read about my Git workflow here.

Assume there’s a dev branch called ‘Fingerboard’ that I need to merge into ‘master.’ The Fingerboard branch has been committed to the Git repo.

git checkout master
git pull origin master
git merge --no-ff Fingerboard

The –no-ff keeps Git from flattening the history for the branch.

git push

Once the merge is complete, the Fingerboard branch is no longer needed.

git branch -d Fingerboard
git push origin --delete Fingerboard

The Perfect Git Workflow

I have concocted the world’s most perfect Git workflow.

Here it is.

Spoiler, it’s actually not perfect, but it works for me.

In Git, there’s the concept of a master branch. Master is merely the first branch Git creates. Git might as well called it “Pippy Pee Pee Diarrheastein Poopypants Esquire.” But Git didn’t do that and so it’s called “master.”

Master, Master, where’s the dreams that I’ve been after? Master, Master, you promised only lies

Master is just a branch like any other branch. You can choose to never use master. I use it. I treat master as the main trunk of the Git tree.

When making a release of code to production, I use master to create the release.

When making codebase changes, I branch master and perform my changes in that branch.

At my current job, the process to roll out to production requires a business owner review the changes.

To do the review, I create a review branch which corresponds in name to what the next release will be called. Therefore, if the next release is 1.2.1, I create a branch of master called review-1.2.1 and merge whichever changes need to be reviewed into it.

Once the review is done, the changes or branches which pass review are merged to master and a release is created for production.

To include more changes in the release to production, the process will start over with a new review branch.

This workflow allows for an unlimited number of dev branches to be created. The review branches gives business owners the opportunity to approve code changes, this keeps stuff out of master that will not go in the next release.

It’s not the most perfect workflow, but it does work for me.

In my ideal world, each change is done in a branch, reviewed quickly and pushed to production as they’re reviewed. But my current process doesn’t allow that, I have to create releases that contain multiple change sets. I think it’s more error prone, but there’s nothing I can do about it.

Pass node arguments to PM2

What I used to do with Perl or shell scripting, I do with Node.js. Here’s how to pass node arguments to a pm2 start command.

I have Node.js application that uses garbage collection. For it to work, I have to pass the –expode-gc node command-line argument. Since I use pm2 to manage my Node.js applications, I needed to send that argument to pm2.

Here’s how to do that.

pm2 start mynodeapp --node-args="--expose-gc"

Cheers.

How to Merge Files Without Using Merge

I use a Mac almost exclusively for development. Mac doesn’t come with the Linux merge command and sometimes I need to merge text files. It’s pretty easy to do using diff and grep.

I’m going to show you how to merge the changes from two files into one.

I’ll use file1, file2, file3 and file4 as the end product.

diff -DVERSION1 file1 file2 > file3

This commend puts all of changes from file1 and file2 into file3. Open file3 in a text editor. You will see stuff like this.

#ifdef VERSION1
// I updated the comment in file, but it doesn't conflict with file2 so it looks like this
#endif

If there is a conflict between file1 and file2, it might look like this.

#ifdef VERSION1
// I updated the comment to fix a typo
#else /* VERSION1 */
// I updated the coment to fix a type
#endif /* VERSION1 */

I then go through file3, searching for ‘#else’ and resolve those conflicts. I generally consider file1 to be the authority, so I usually remove the #else leaving the #if part.

Then when I’m done resolving conflicts, I run this.

grep -v '^#if' file3 | grep -v '^#endif' > file4

Now file4 has all of the correct changes from file1 and file2.

Edit Eclipse Generate Getter and Setters Format

For me, I like to always return $this when creating a setter class method when coding PHP.

For example, to set an object to be freaky, I would create method like this.

/**
 * Set to freaky or not freaky.
 *
 * @param boolean $freaky
 * @return $this
 */
public function setFreaky($freaky) 
{
    $freaky === TRUE ? $this->freaky = TRUE : $this->freaky = FALSE;
    return $this;
}

Then if I had a method to set an object’s hair color, I could have a method like this.

/**
 * Set hair color
 *
 * @param string $color
 * @return $this
 */
public function setHairColor($color) 
{
    $this->color = $color;
    return $this;
}

Now if I can chain these method calls.

$dreamPartner = new Partner();
$dreamPartner->setFreaky(TRUE)->setHairColor('#FF3333');

Not everyone may prefer this technique, but I like it better than.

$dreamPartner = new Partner()
$dreamPartner->setFreaky(FALSE);
$dreamPartner->setHairColor('#FFF5DD');

There’s no right or wrong answer, but returning $this in setters, gives the programmer using the class the option of chaining method calls. There’s no downside to doing this.

If you’re using Eclipse, you can modify the “Generate Getters and Setters” formatting for PHP under Preferences. Look for PHP -> Code Style -> Code Templates. On the Code Templates screen, expand Code and select Setter Body.

Double-click Setter Body and it should look like this.

${field} = ${param};

I changed it to look like this.

${field} = ${param};
return $$this;

How to Make Eclipse IDE Faster

Eclipse Suspend all Validators While coding the other day, Eclipse was running like shit. It was slow as hell. I couldn’t get anything done. I did all of the updates to Eclipse and even rebooted. It didn’t it help, it was dog ass slow.

Checking my laptop resources, it was all fine. Researching this issue, I found out about a setting to suspend validators in Eclipse.

You can find the setting under preferences and Validation. You should see a checkbox to “Suspend all validators.” Once I did that, Eclipse was back to its normal self. Since upgrading to Oxygen, I’ve had a lot of issues that I didn’t use to have. I don’t think I need the validators on, so it doesn’t matter to me if they’re suspended.

Configure Composer to Allow for dev-master

Composer

By default composer doesn’t allow you to install packages that are dev-master, but it’s easy to change that behavior.

When you install a package that only has a dev-master version and it fails, you need to update your composer.json.

Composer error

It’s easy to fix this composer error.

In your project’s composer.json, set “minimum-stability” to “dev.”

Your composer.json might look like this.

{
    "name": "My Poopy Application",
    "minimum-stability": "dev"
}

Now when you want to add package’s dev-master version, you can do so.

Could not find a matching version of package hack3r3d/chester. Check the package spelling, your version constraint and that the package is available in a stability which matches your minimum-stability (stable).

PHP Portable Paths

Forward SlashFrankly, I don’t care about people wanting to run my code on a Windows box and it not working. But in reality, for some reason, there are people who write code using computers running the Microsoft Windows operating system. Why don’t they use a Mac, or if they don’t have or want to spend the money on a Mac, use Linux? That I do not know, but there are people, and they probably work in your company who use Windows and they think they can code.

Like I said, I don’t really care if my shit doesn’t run on Windows, but it is relatively trivial to write PHP code that can run on Windows and Linux, Mac and any operating system that can run PHP.

The only real obstacle to portability is directory paths. In my world, these are usually paths to configs or log directories.

The could be code like this.

$config = new Configuration(sprintf('conf/%s', $configFilename));

That’s not portable to Windows because of that forward slash.

What I could do is this.

$config = new Configuration(sprintf('conf%s%s', DIRECTORY_SEPARATOR, $configFilename));

Now instead of a forward slash, if this were running on Windows, it would use a backward slash, because Windows is backwards.

I took it one step more and wrote a static method in my Configuration class that looks like this.

public function static Path($path)
{
    if (DIRECTORY_SEPARATOR == '/') return $path;
    return str_replace('/', DIRECTORY_SEPARATOR, $path);
}

This method will replace all forward slashes with DIRECTOR_SEPARATOR. If the DIRECTOR_SEPARATOR is a forward slash, the path is just returned unchanged. If DIRECTOR_SEPARATOR is not a forward slash, forward slashes in path are replaced with what is DIRECTOR_SEPARATOR.

I do it this way because in my code, I still put the forward slash in. The default is that my code will be running on some version of Linux, so the forward slash is correct. But since I pipe my paths through Configuration::Path, if they need to be changed, the directory separator will be changed to the operating system’s appropriate separator.

So in my code, using the Configuration Path static method, I would do something like this.

$config = new Configuration(Configuration::Path(sprintf('conf/%s', $configFilename)));

This pathing issue alone doesn’t make your code automatically 100 percent portable. There are issues with specific PHP functions that may require different things to happen when your code is running on Windows or Linux. But the pathing issue is the easiest to do correctly.

How to Configure Apple Airport Extreme and Verizon Fios Router

I prefer Apple products over pretty much any other computer product any day of the week – if I can afford it. Prior to getting Verizon Fios, my preferred wifi router was my Apple Airport Extreme. What I like about the Airport Extreme is that I can plug a hard drive into it and do automatic Time Machine backups whenever I’m connected to my wifi network.

Here’s how my network is setup. I have two wifi networks, once provided by the Fios router and the another by my Apple Airport Extreme. The Airport Extreme connects directly to the Fios router via Ethernet.

I can not to the Airport Extreme base station. Everyone else connects to the Fios one. That limits the number of devices connecting to the Airport Extreme that has my hard drive connected to it. I don’t have to share that password with house guests. And it also splits the workload between these two routers, which really do serve different purposes in my home.

You’ll see a lot of discussion about disabling the Fios router wife network. I didn’t see why that would be necessary. If you don’t need it, it makes sense to turn it off, but I don’t think it hurts to keep it around for a guest network.