Invalid Certificate after Security Update 2015-004 in Mavericks

After recently installing Security Update 2015-004, I found that I could no longer browse to any website using the root certificate “VeriSign Class 3 Public Primary Certification Authority – G5” without a security warning (“invalid certificate”). This included sites such as Twitter and Apple, and it also meant that applications such as Software Update would no longer function.

After digging into it (see here, here, here and here) I found the cause was a chain of events that while a bit convoluted, were fairly prevalent among users.

First off, 2015-004 updated the list of trusted root CAs which by itself isn’t an issue. The problem was when I then logged into Amazon S3 using an older version of Cyberduck (< 4.7). That version of Cyberduck was adding the certificate chain retrieved from Amazon to my login keychain which also by itself isn't an issue. The problem was that the intermediate certs Amazon was using were outdated and signed with 1024bits. This caused a mismatch between the certs installed by 2015-004 and the ones being saved to the keychain by Cyberduck. Like I said, convoluted.

Luckily everyone seems to have implemented fixes – Cyberduck no longer writes the intermediate certs to the keychain (as of version 4.7) and Amazon has updated their intermediate certs to 2048bit signatures.

If you run into this issue, you probably still have the invalid certs sitting in your keychain. Simply open up Keychain Access and delete the bogus entries in the login keychain so that the system entries are used instead (select login, then Certificates, you should see them at the bottom of the list – "VeriSign Class 3 Public Primary Certification Authority – G5").

Django, Haystack and Elasticsearch – Part 1

I’m wrapping up a little side project at the moment (more on that very soon) which required full-text search, autocomplete, and a few other bits of search related functionality.

After some research I landed upon the combination of Elasticsearch and the awesome Django application Haystack.

First step was to get Elasticsearch up and running locally on OS X…

1) Download latest zip from http://www.elasticsearch.org/overview/elkdownloads/. A good spot is:

/opt/elasticsearch-1.1.x

2) Create the following directories:

/opt/elasticsearch-1.1.x/data
/opt/elasticsearch-1.1.x/work
/opt/elasticsearch-1.1.x/logs

3) Add the following to your .profile (allows you to run Elasticsearch from the command prompt without the full path):

# elasticsearch
export ES_HOME=/opt/elasticsearch-1.1.x
PATH=$ES_HOME/bin:$PATH

view raw
.profile
hosted with ❤ by GitHub

4) Update the following values in the Elasticsearch config file:

# /opt/elasticsearch-1.1.x/config/elasticsearch.yml
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
cluster.name: elasticsearch
network.host: 127.0.0.1
http.port: 9200
path.conf: /opt/elasticsearch-1.1.x/config
path.data: /opt/elasticsearch-1.1.x/data
path.work: /opt/elasticsearch-1.1.x/work
path.logs: /opt/elasticsearch-1.1.x/logs

view raw
elasticsearch.yml
hosted with ❤ by GitHub

5) Ensure all requirements are installed (django-haystack, pyelasticsearch, requests, simplejson):

pip install django-haystack
pip install pyelasticsearch

view raw
install.sh
hosted with ❤ by GitHub

6) You should now be able to start Elasticsearch:

$ elasticsearch
[2014-05-14 08:15:05,257][INFO ][node ] [Aminedi] version[1.1.1], pid[46224], build[f1585f0/2014-04-16T14:27:12Z]
[2014-05-14 08:15:05,258][INFO ][node ] [Aminedi] initializing …
[2014-05-14 08:15:05,271][INFO ][plugins ] [Aminedi] loaded [], sites []
[2014-05-14 08:15:07,136][INFO ][node ] [Aminedi] initialized
[2014-05-14 08:15:07,136][INFO ][node ] [Aminedi] starting …
[2014-05-14 08:15:07,211][INFO ][transport ] [Aminedi] bound_address {inet[/127.0.0.1:9300]}, publish_address {inet[/127.0.0.1:9300]}
[2014-05-14 08:15:10,262][INFO ][cluster.service ] [Aminedi] new_master [Aminedi][X4diAes4TrOMTk4eAdbhnA][mbp.home][inet[/127.0.0.1:9300]], reason: zen-disco-join (elected_as_master)
[2014-05-14 08:15:10,284][INFO ][discovery ] [Aminedi] elasticsearch/X4diAes4TrOMTk4eAdbhnA
[2014-05-14 08:15:10,298][INFO ][http ] [Aminedi] bound_address {inet[/127.0.0.1:9200]}, publish_address {inet[/127.0.0.1:9200]}
[2014-05-14 08:15:10,784][INFO ][gateway ] [Aminedi] recovered [1] indices into cluster_state
[2014-05-14 08:15:10,785][INFO ][node ] [Aminedi] started

view raw
elasticsearch.sh
hosted with ❤ by GitHub

7) Add Haystack to your Django config:

# add to installed apps
INSTALLED_APPS = (
'haystack',
)
# haystack search using elasticsearch
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://127.0.0.1:9200/&#39;,
'INDEX_NAME': 'haystack',
},
}
# http://django-haystack.readthedocs.org/en/latest/signal_processors.html
HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'
# increase the default number of results (from 20)
HAYSTACK_SEARCH_RESULTS_PER_PAGE = 40

view raw
config.py
hosted with ❤ by GitHub

8) After you’ve added your search indexes, you can use manage.py to rebuild the search index:

$ python manage.py rebuild_index

Connecting to a Local Django Server from VMware Fusion on OS X

Even though we’d all like to believe that IE is dead and there’s no need to test it any more, that simply isn’t the case yet. But debugging via a remote server is a pain when you can easily use a local Windows VM running on VMware Fusion.

The following will get you up and running with a local Django server (running on port 8000) and any Windows OS. Note that I’m using the latest VMware Fusion (version 6) but the same steps will work with previous versions as well.

First, shutdown the VM (power it off completely) and then open it’s settings. Click Add Device, select Network Adapter and click Add.

Now you’ll see 2 network adapters for the VM. Click the new one to edit it.

Switch the connection type to Private to my Mac.

Next, grab the IP address your Mac is using for this private network. Open a terminal window and run “ifconfig vmnet1”.

$ ifconfig vmnet1
vmnet1: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ether 00:50:56:c0:00:01
inet 172.16.85.1 netmask 0xffffff00 broadcast 172.16.85.255

The last line is the important one. Make note of the IP address next to inet, in this case mine is 172.16.85.1. Because we made a new private network, this IP should hopefully stay the same and you won’t have to worry about messing with the configuration again down the road.

Next, start your Django server using the IP address 0.0.0.0:

$ python manage.py runserver 0.0.0.0:8000

Now start the VM back up and open up IE or your browser of choice. Go to http://172.16.85.1:8000 (make sure to substitute the IP address you made note of above). Voilà, you’re browsing your local Django server from Windows.

If you’d like to use something more memorable for the host name, go ahead and edit the Windows host file (it can be found here – C:\Windows\system32\drivers\etc\hosts). Add the new host to the list of entries:

127.0.0.1      localhost
172.16.85.1    mysite.local

I found that xxxx.local worked reliably while other host names were hit and miss.

Finally, flush the DNS on your VM for the host changes to take effect:

ipconfig /flushdns

You can now reach your Django site at http://mysite.local:8000.

Sleepy Thunderbolt Display

Ran into another Mavericks upgrade issue today – my Thunderbolt display would no longer wake up (MacBook Pro connected, cover closed, keyboard & mouse attached to the monitor).

Quick fix is to reset the System Management Controller (SMC):

http://support.apple.com/kb/HT3964?viewlocale=en_US

MacPorts & Mavericks OS X 10.9 & Gnutar Errors

UPDATE – The 10.9 Mavericks binary is now available here.

If you try installing or updating MacPorts ports after upgrading OS X to 10.9, you’ll hit an error with gnutar:

mbp:tar-1.27 chriskief$ sudo port upgrade outdated
> Extracting apr
Error: org.macports.extract for port apr returned: command execution failed
Please see the log file for port apr for details:
/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_apr/apr/main.log
Error: Unable to upgrade port: 1
To report a bug, follow the instructions in the guide:
http://guide.macports.org/#project.tickets

view raw
macports.sh
hosted with ❤ by GitHub

:debug:extract Assembled command: 'cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_apr/apr/work" && /usr/bin/bzip2 -dc '/opt/local/var/macports/distfiles/apr/apr-1.4.8.tar.bz2' | /usr/bin/gnutar –no-same-owner -xf -'
:debug:extract Executing command line: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_apr/apr/work" && /usr/bin/bzip2 -dc '/opt/local/var/macports/distfiles/apr/apr-1.4.8.tar.bz2' | /usr/bin/gnutar –no-same-owner -xf –
:info:extract sh: /usr/bin/gnutar: No such file or directory

view raw
macports.log
hosted with ❤ by GitHub

The recommended fix from MacPorts is to reinstall MacPorts and all ports after a major OS upgrade. Unfortunately there is no binary for Mavericks yet and quite a few ports are failing to build right now including MySQL 5.6 and Python 2.7.

The  workaround is to keep your current MacPorts installation and install gnutar from source:

wget http://ftp.gnu.org/gnu/tar/tar-1.27.tar.gz
tar xzvf tar-1.27.tar.gz
cd tar-1.27
./configure
make
sudo mv ./src/tar /usr/bin/gnutar

view raw
install.sh
hosted with ❤ by GitHub

This is definitely not a long term solution and should really just be used for new ports that are must haves. I wouldn’t run port upgrade outdated or try to upgrade any specific ports as you will find that not all ports will build.

Once a Maverick’s binary is out, be sure to remove this gnutar before doing the migration.

Read more discussion of MacPorts on Mavericks.

Note: Be sure to build and install gnutar from source and not just symlink…

OS X Mavericks, Xcode 5.0.1 and the Missing Command Line Tools

So not only was Java gone from my system after the update, the Xcode command line tools disappeared as well. I’m not sure if they disappeared with the Mavericks update or when I upgraded Xcode to 5.0.1, but regardless, they’re gone.

In the past few versions of Xcode you could open the application, head to preferences, and select them under Downloads > Components. Unfortunately they’re no longer there.

In classic Apple fashion they’ve changed the way you get the command line tools without really mentioning it (at least not that I could find). Most likely because there were many folks who wanted the tools without having to install Xcode.

There are now two ways to get them:

1) Download from the Apple Developer Site (you’ll need to login with your account)

2) Or trigger the install to start by running the following command: xcode-select --install

And to be extra clear, Xcode is no longer required in order to install the command line tools as described above.

OS X 10.9 Mavericks, Chrome and Java

UPDATE: As of version 39, Chrome is now 64-bit! You can now simply download the latest version of Java and install. No more having to re-enable older versions of Java.


After upgrading to Mavericks I noticed that Java was no longer present on my system. Java-based applications such as PyCharm would no longer run and browser-based applets would display a plugin missing error in both Safari and Chrome. To get up and running again, two installs needed to happen.

First was the runtime. You can download Java for OS X 2013-005 directly or the OS will automatically download and install it if you launch a Java-based application such as PyCharm.

Once installed, $ java -version returns 1.6.0_65:

The second step was to install Java 7 from Oracle. Unfortunately this version of Java is only compatible with 64-bit browsers such as Safari. To use Chrome (still 32-bit), you must then disable Java 7 and re-enable the Apple-provided Java SE 6 web plug-in by following these simple steps.

A Tale of Two Upgrades – OS X 10.9 vs Windows 8.1

Yesterday marked the release of Mavericks, aka OS X 10.9, so I figured I might as well upgrade both the MacBook and my Windows 8 VM and see how the process goes with the two different platforms. First up, OS X.

The install was beyond easy – opened up the app store, clicked the install button, watched it download like any other app, and clicked install once the download was finished. About 40 minutes later I was running 10.9.

Along the way there was plenty of feedback including how long the download was going to take, progress bars during the installation with estimated completion times, etc. At no point was I wondering what was going on or how long until it was going to be finished. The upgrade even let me know that there was an incompatible application, just one, that couldn’t be used with Mavericks and that it had moved it to a folder for me.

Now onto Windows 8.1. All I can say for this upgrade is at least Microsoft figured out how to avoid mailing me an installation DVD. I should have Googled before I started as there are heaps of articles, posts and what not regarding the frustration filled upgrade process.

Things started off badly from the very beginning as the update did not even appear in the Windows Store. A little searching revealed the following gem:

Although the Windows 8.1 update is downloaded and installed from the Store, your PC needs another update, KB 2871389, before Windows 8.1 becomes available to you in the Store.

Of course, update before upgrading. This is Windows. Should have known.

Once I had the upgrade appearing in the store, I proceeded to click the install button. Some spinning balls animated around the screen and then poof, I’m back to the main Windows Store page. No progress bar, no indication anything is happening, nada. Some more Googling revealed that I’m not the only one to not see anything and that the upgrade is in fact downloading. Okay great, I’ll just wait.

After about 30 minutes (the store said it was about 3GB) a new screen appeared telling me I have to reinstall my applications once the upgrade is complete. That sounded a bit ridiculous but a quick search verified it:

You can download the update for free from the Windows Store. You’ll be able to keep your personal files, but you’ll need to reinstall your apps. Make sure you have any original installation media that might be required for apps that were not installed from the Windows Store.

This is a VM with minimal applications so not the end of the world, but if this was my main machine…

After clicking ok that screen disappeared and again I was back on the Windows Store homepage. Now what the hell is going on? I let it sit for another 10 minutes or so and then I was finally prompted to restart to install 8.1.

After rebooting the installer did it’s thing for a while, took me through a few setup screens (pick a color was the first one… is that really necessary?), and I was finally running 8.1. As warned my applications were gone so they were right on that one… or wrong, if you stop and think about it. At least my experience went better than this.

A few hours and a few GBs later I’ve got two upgraded OS’s that both look and function extremely similarly to their predecessors (kudos to Apple for pushing me a desktop alert to this page). I’m very curious to see what response Microsoft has for Apple’s $0 price point for 10.9.

While this may sound like a Microsoft bashing, that wasn’t the intention. I really hope they can continue to compete with their software. They’ve got thousands of talented engineers making some of the most widely used, and profitable, software products in the world, so there’s no reason they shouldn’t be able to.

But with all that talent and money, they still can’t figure out how to polish up their user experience. Simple things like progress bars on downloads and prompts for missing updates (i.e. show an 8.1 tile in the store that tells the user to update their computer) would go a long ways to ease consumer frustration.

My real fear however, is that the issues actually run much deeper. At it’s core, Windows is now a weird hybrid of an OS. Part tablet interface, part traditional desktop. It’s like they built it for a world where all desktops were touchscreens. Unfortunately that world doesn’t appear to be materializing (perhaps why Steven Sinofsky is no longer working there). And they’re now left rationalizing this Frankenstein OS with weird arguments like this:

Shaw asks rhetorically, what’s harder: clicking on a button in Windows that switches between a tablet and desktop mode, or closing a laptop, then picking up a tablet, turning it on for certain apps, then closing the tablet and putting it away and grabbing a desktop for a desktop activities? In Microsoft’s opinion, having it all in once device makes more sense than owning two devices.

I’m pretty sure no one has actually wanted to use tablet apps on a desktop using a mouse. Ever.

And I think Microsoft knows it. They’re simply unwilling to admit that after nearly destroying Windows with Vista, and the subsequent resurrection with Windows 7, they’ve completely blown it again with 8. Luckily they’ve got the time and market share to course correct once again. I just hope they come to their senses sooner rather than later.

My guess is we’ll see Windows 9 return to a traditional desktop OS with a touch version for phones and tablets. I’d even encourage Microsoft to change the name since the Windows metaphor just isn’t appropriate any longer. Just call it Tile or something simple like that (not Windows Tile or Windows Tile RT, just Tile for peat’s sake).

Those changes along with continued simplification of their software (more on this in another post) may be just what Microsoft needs to feel innovative and competitive again.