Symfony Doctrine To Flush or Not to Flush

I’m in the process of converting an old MySQL database to a shiny new MySQL database controlled via Doctrine. Some of the tables have tens of thousands of records in them, so I was curious if it made sense to flush after each insert, wait till the end and flush or flush periodically during the process.

What I found was that waiting until the end to flush 200,000 records didn’t really work, it took forever and I never did see it complete. Flushing after every insert into the new DB was slow as a hell, but it did work. I came down to just flushing every 1,000 records and that seems to work great.

So in my PHP code that’s handling the conversion (Symfony Console Component), I run a counter and check $counter % 1000 == 0, I flush, then I flush when it’s all over to catch the stragglers.

If you’re using Symfony, don’t try to do this sort database conversion via a controller and a web page, it simply doesn’t work. Use Symfony’s Console Component bundle, it works great.

Mounting Shared Drive on VirtualBox Ubuntu Guest Fails “device not found”

Screen Shot 2016-07-15 at 11.04.13 PM So here’s a weird thing about VirtualBox that kind of bit me in the ass twice in like a week, so I decided to document it here so I won’t get burned again.

I use VirtualBox to run Ubuntu as my LAMP stack for development. Configuring a proper LAMP stack on Mac is a pain in the ass and never worth the hassle. I used to use MAMP, which was fine, but it doesn’t work with modern versions of openssl, so it doesn’t work with Paypal and shit like that (older version of TLS have been retired).

I do my development on my Mac and then I share the drive where my code resides with the VirtualBox Ubuntu guest. But if you try to setup up share on the Vbox host to be called “Documents,” for example, and you want to mount it on your guest as “Documents,” it will fail. I have no idea way. It may actually be that the share name on the host can’t be the same as the folder you’re sharing. I’m not sure, I didn’t test that.

So this will not work if the host machine share folder is also named Documents.

sudo mount -t vboxsf -o uid=1000,gid=1000 Documents /media/Documents

You have to name your share something different, like MyDocs, or you get “device not found.”

But this will work.

sudo mount -t vboxsf -o uid=1000,gid=1000 MyDocs /media/Documents

Also note that on the guest, you have to have the Virtual Box guest extensions install to be able to mount vboxsf drives.

And if you want it to mount on boot, you need to put this in fstab.

MyDocs          /media/Documents        vboxsf  rw,uid=1000,gid=1000    0       0

Now you may have an issue with this auto mounting on boot because the share might not be there when Ubuntu needs it to be there on boot.

One option is to add “vboxsf” to you /etc/modules file. That forces the vboxsf stuff to load earlier during boot. That totally worked for me.

The other option, which also worked for me, and seems a bit less hackish, is to add the _netdev options to the fstab line so it looks like this.

MyDocs          /media/Documents        vboxsf  rw,uid=1000,gid=1000,_netdev    0       0

Also don’t check Auto-mount when you create the share on the VirtualBox host, it mounts the drive in /media/sf_MyDocs and it makes it all owned by root.vboxsf. I don’t like that, but it can work for some people. If you mount it yourself in Ubuntu, you have more control.

So there you have it. Now you should be able to mount your shared drive on your VirtualBox Ubuntu guest on boot.

Don’t forget those guest extensions on the guest machine.

sudo apt-get install virtualbox-guest-utils virtualbox-guest-additions-iso

How to use Symfony’s EntityType to Make a Dynamic Dropdown List

Making a form element in Symfony be populated by a list of entities isn’t too hard, but it’s a got some quirkiness to it.

To do this you need the following.

  1. An entity that has more than one of it stored in your database.
  2. A form class.
  3. An entity repository class.

First create your form class like this.

$builder->add('beer', EntityType::class, array(
  'label' => 'Beer',
  'class'=>'MyBundle:Beer',
  'property'=>'name',
  'query_builder'=>function(EntityRepository $er) {
         return $er->getBeers();
   }))
   ->add('drink', SubmitType::class, array(
         'label' => 'Drink this beer'
   ));

Then create a Beer entity that has an attribute “name.” Update your database and add a few beers to the table. If you don’t know how to do this, check out the Symfony documentation on creating and retrieving entities from a database.

Now create BeerRepository class. It should live alongside your Beer.php entity class in the “entity” directory.

In the BeerRepository, create the method mentioned above – getBeers.

public function getBeers()
{
  $query = $this->getEntityManager()->createQueryBuilder()
        ->select('b')
        ->from('MyBundle:Beer', 'b')
        ;
   return $query;
}

This method builds a query that the form will use to pull all of the beers out of the database and display them in a drop down list.

There’s one thing that tripped me up. It’s that part of the form about “property.”

'property'=>'name'

What that is saying is, in the dropdown, display the beer name. I knew there had to be way to tell the form which field to use, but it took forever for me to figure out it was this property attribute.

You could also define a __toString method in the Beer entity class that looks like this.

public function __toString()
{
  return $this->name;
}

If you do it in the entity, it’s the same for every time this entity is used in a dropdown or radio button context. If you do it in the form, you can change it on a form-by-form basis. I guess the form setting would override the __toString, but I haven’t tested it. I like doing it in the form builder, it seems appropriate to be there as it is really related to the form creation.

To change it from a dropdown to radio button, add ‘expanded’=>TRUE to the form builder. If you want the use to be able to select more than one option, like with checkboxes, add ‘multiple’=>TRUE. And while we’re at it, if you set expanded to FALSE and leave multiple to TRUE, you will get a box listing all of the beers, allowing you to select more than on.

As always, drink at home, or drink safely.

How to Customize Symfony Form Widget Error Messages When Not Using an Entity

Earlier today I was researching how to customize error messages in a Symfony form type that isn’t linked to an entity.

Initially I wanted to do this in the form builder class, but I couldn’t find the solution. Then I found a solution to do what I need in Twig and that was fine.

But now I’ve found the correct solution, the one that works regardless of where my form is rendered in my application. You can modify the error message in the form builder, it’s actually quite simple, I don’t know why I couldn’t find this earlier today.

$builder->add('firstname', TextType::class, array(
      'label' => 'First Name',
       'required' => TRUE,
       'constraints'=>new NotBlank(array('message'=>'Please enter a first name'))
))

This is the more correct way to do this when not using an entity. But my other solution is nice and dirty, and requires less work in some instances, but it’s less portable.

To close the loop on this one, I actually did figure out how to put my error messaging in the form builder, which in this case, seems appropriate.

For the credit card expiration date, for instance, I’m doing this.

->add('ccexp', TextType::class, array(
        'label' => 'Card Expiration',
        'required' => TRUE,
        'constraints' => new Regex(array(
            'pattern' => '/^(0[1-9]|1[0-2])\/|\-?([0-9]{4}|[0-9]{2})$/',
            'message' => 'The credit card expiration date is not valid. Enter MM-YYYY, such as 01-2020 for January 2020.'
        ))

Customize Symfony Form Builder Error Message in Twig Template

I’m building a form to gather credit card information and I want to customize the error messages displayed on the form.

I’m sure I can do this with form theming, and that’s great, but I just wanted a down and dirty, in the template, spitting out custom error messages solution. Of course, normally, I would put my entity validation stuff in the entity class, but in this case, the form was not tied to an entity. I’m not actually saving the credit card information, I’m sending it to Paypal.

It took a bit of digging to solve this problem. I assumed that I could just specify an error message when building the form in my form builder class, but apparently that’s not the case.

So here’s how I did it for the credit card expiration field in the Twig template.

<fieldset>
{% if form.ccexp.vars.errors|length >0 %}
Oops, the credit card expiration date is invalid, please specify the expiration date in this format mm/YYYY (e.g. 1/2020 for January 2020).
{% endif %}
{{ form_label(form.ccexp) }}
{{ form_widget(form.ccexp, {attr: { 
        'data-format': "MM-yyyy" },
    }) }}
</fieldset>

PHP Soap Server Repeat the Same Element

In a SOAP service I work on, I needed to generate a response that repeated the same element.

Something like this.

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://www.example.com/WebService/">
   <SOAP-ENV:Body>
      <ns1:getPhoneNumbersResponse>
         <numbers>
            <number>123341344</number>
            <number>329822938</number>
         </numbers>
      </ns1:getPhoneNumbersResponse>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Getting this to work in the PHP code took a bit of fussing around. You can’t just create a hash, because the hash can’t include more than one key called ‘number.’

$this->response = new GetPhoneNumbersResponse();
$numbers = array();
$number1 = new \SoapVar('123341344', XSD_STRING, NULL, NULL, 'number');
$numbers[] = $number1;
$number2 = new \SoapVar('329822938', XSD_STRING, NULL, NULL, 'number');
$numbers[] = $number2;
$this->response->numbers = new \SoapVar($numbers, SOAP_ENC_OBJECT, NULL, NULL, 'numbers');

In my wsdl, it looks something like this.

<xsd:element name="getPhoneNumbersResponse">
  <xsd:complexType>
    <xsd:sequence>
      <xsd:element name="numbers" maxOccurs="1" minOccurs="1" type="tns:getPhoneNumbersResultInformation"></xsd:element>
    </xsd:sequence>
  </xsd:complexType>
</xsd:element>
<xsd:complexType name="getPhoneNumbersResultInformation">
  <xsd:sequence>
    <xsd:element name="number" maxOccurs="unbounded" type="xsd:string" />
  </xsd:sequence>
</xsd:complexType>

The key is that in the PHP code, I had to use the SoapVar object and force the data to be in the correct tag.

$number1 = new \SoapVar('123341344', XSD_STRING, NULL, NULL, 'number');

That makes a “number” element.

I then assign it to an array $numbers[]; Once that array has all of the ‘number’ SOAP strings, I then add it to my response object and put it in the numbers tag.

$this->response->numbers = new \SoapVar($numbers, SOAP_ENC_OBJECT, NULL, NULL, 'numbers');

Also note that “number” is a string (XSD_STRING) and “numbers” is cast as a SOAP object (SOAP_ENC_OBJECT).

I’ve actually had to solve this a few times, and each time I forget how to do it, so hopefully I’ll remember that I wrote this down.

You may see reference on Stackoverflow for using the wsdl array stuff, I played around with that, but this approach documented here is more straightforward.

How to Configure Redis to Automatically Start when Mac OSX Boots

I don’t like to remember to do stuff, because I invariably forget. I’ve recently started using Redis for a couple of projects and so I had to install it on my Mac for development. Here’s what I did to make Redis start on boot.

I didn’t use homebrew to install Redis, I compiled version 3.2.0 from the source. It’s actually super easy. It’s just “make,” “make test” and then “sudo make install.” Doing so installs Redis in /usr/local/bin.

Compiling from source doesn’t get you a redis.conf file, but I just used the default example I found on Redis.io. I saved it in /usr/local/etc. If you’ve installed things in a different location, adjust the following instructions to match your Redis setup.

To make Redis automagically startup on boot, I’m using launchd.

Create a simple xml document in /Library/LaunchDaemons.

sudo vim /Library/LaunchDaemons/redis-server.plist

In redis-server.plist put this.

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>redis-server</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/redis-server</string>
        <string>/usr/local/etc/redis.conf</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
</dict>
</plist>

To repeat, if you don’t have redis-server installed in /usr/local/bin, you need to change your path in the xml file. Also, if you don’t use a redis.conf, or it’s not installed in /usr/local/etc, update accordingly. If you don’t have a redis.conf, you can just remove it from this xml file.

Now load the redis.server.plist into launchd.

sudo launchctl load /Library/LaunchDaemons/redis-server.plist

You only have to do this once, now you should have control of your redis server via launchd.

Fire up Redis with this command.

sudo launchctl start redis-server

If you did it correctly, crank up the redis-cli client and type “ping.” You should connect and receive a “PONG” response. If you didn’t, go to /usr/local/bin and try running “sudo ./redis-server.” It should start up. If it doesn’t, fix whatever error is being generated. If it did work, now try it with the redis.conf file: “sudo ./redis-server ../etc/redis.conf.” If this doesn’t start Redis, fix any errors reported. If it does start, run redis-cli again and try the “ping” test.

How to Make a SOAP Request with Node.js using raw XML

There are libraries out there that you can use to make SOAP requests from Node.js. But I really just wanted to throw raw XML at my SOAP service. None of the libraries seemed to be able to do that, so I’m using the Node.js http implementation.

var http = require('http');
var http_options = {
  hostname: 'localhost',
  port: 80,
  path: '/LocationOfSOAPServer/',
  method: 'POST',
  headers: {
    'Content-Type': 'application/x-www-form-urlencoded',
    'Content-Length': xml.length
  }
}

var req = http.request(http_options, (res) => {
  console.log(`STATUS: ${res.statusCode}`);
  console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
  res.setEncoding('utf8');
  res.on('data', (chunk) => {
    console.log(`BODY: ${chunk}`);
  });

  res.on('end', () => {
    console.log('No more data in response.')
  })
});

req.on('error', (e) => {
  console.log(`problem with request: ${e.message}`);
});

// write data to request body
req.write(xml); // xml would have been set somewhere to a complete xml document in the form of a string
req.end();

This code is almost exactly what’s in the Node.js documentation. Obviously you can send any sort of POST data to any web server, or submit GET requests. Node.js uses this http implementation rather than what Javascript uses, which is XMLHttpRequest.