macOS 10.15 vs Windows 10: Apple is losing ground

    After a few discussions and my previous blog post, I've noticed that I've changed my tenor on macOS, by far my most loved Apple product. I used without question profess macOS as the superior OS* (aside from gaming). Unfortunately, this post is going to feature some serious nerdage without a lot of explination, for the sake of brevity.

    Now that asterisk has expanded to include a lot more things. Take the following: I cannot properly stream a 4k movie 1 in surround sound 2 in HDR 3 on an Nvidia GPU 4 or be sure it's using hardware decoding 5.

    Let me break that down:

    1. macOS currently does not support the latest DRM that Netflix most other DRMed streaming services use. macOS is capable of playing back 4k media. Windows can play back 4k Netflix in Edge and its app. Vudu and UltraFix also support 4k Windows playback. Windows users face the same issue as Mac users with Amazon Prime and Hulu.
    2. macOS still does not have any multichannel decoding for popular consumer codecs from Dolby or DTS. Even the Apple touted AAC can do multichannel audio, but macOS has no ability to decode to multichannel outputs. macOS can route multichannel audio via CoreAudio using prosumer/professional hardware, but to this day, no front end for software exists to route multichannel audio to analog outputs. It can, however, pass pre-existing bitstreams via SPDIF and HDMI in a few applications, most notably VLC and not any Apple software.
    3. 10-bit support for macOS is completely bonkers. My MacBook Pro 15 inch 2017 has a Radeon Pro 555x and reports 10-bit on the internal display. The internal display is not 10 bit, but rather 8 bit. My Mac Pro 2010 has a Vega 56 connected to a BenQ PD3220u (a true 10 bit panel, not FRC) reports 10-bit, but it's unclear when/if 10 bit is actually being piped to the display as I can't seem to read the bitstream info. Mac OS does not any consumer formats for HDR support regardless. Windows 10 will automatically in games trigger the HDR color profile on the monitor as it detects the HDR10 bitstream. macOS never even does this. By visual tests, it appears my MacBook will output 10 bit to the monitor. The Vega 56 will not but reports otherwise.
    4. Apple, as of 10.13 has actively blocked NVidia support for any non-Keppler GPU. Want an RX 2080 for your Mac Pro 2019? It'll only work in Windows. The spat has drawn widespread media coverage outside the Mac universe. Theories range from eGPU support to Nvidia and Apple stalemating over CUDA. Whatever the case, Mac users are severely handicapped in GPU choice.
    5. Apple no longer officially supports hardware decoding for certain codecs for non-T2 chipset enabled computers. My MacBook Pro 2015 in Catalina now uses the T2 chipset to assist with H264/H265. My Mac Pro, despite having much better hardware without the power consumption requirements, does not. It is unclear if the Mac Pro 2019 can use its $5000 GPU options to assist with these codecs.

    I do not like Windows 10's UX (Why does it have two sets of control panels still!?!?) and inborn advertising, but you gotta hand it to MS. I can run some programs written for Windows 98 using compatibility mode, any 32-bit apps, and it comes with pretty good threat detection with Windows Defender.

    Apple did something though that MS never had to clear two nearly impossible hurdles with the Macintosh platform. The first, switching from Mac OS 9 to *nix, and jumping x86 to PPC. This meant to a few less-than-ideal solutions like Carbon, which eased Mac OS 9 to Mac OS X. That said, x86 Cocoa Apps (Carbon's successor) from 10.6 often do not work in macOS 10.15. Apple does not offer a compatibility mode. Apple also dropped 32-bit binary support in 10.15 arbitrarily, meaning old apps are even less likely to work. This is not a confidence builder for a platform's robustness.

    As far as threat detection, Apple's "security" almost entirely comes from negatively sandboxing the user in 10.15 and over-reliance on signed code, which ironically puts power users at greater risk, as they're the most likely to disable system-integrity-protection. It's evidenced by Malwarebytes' report that Mac malware outpaced Windows PCs threats for first time in 2019. The T2 chipset has some interesting features, and eases things like disk encryption but also is wantonly silly for the most common threat vectors and gives less-educated users a false sense of security. Apple has gone to excessive lengths recently to iOSify its security by depreciating services like Kexts but still has yet to offer threat detection. The Mac Pro 2019 cannot use 3rd party SSDs in its two factory SSD slots because of the T2 despite plenty of M.2 adapters existing to Apple NGFF. If anything, the T2 chipset feels like kneecapping macOS with self-destruct button it can hit at any time, locking users out of their own hardware.

    Lastly comes the most controversial statement, but macOS is not the fastest OS. You can take your pick of benchmarking site, but macOS on the same exact hardware routinely is bested by Windows (and, of course, Ubuntu). It's tough comparing benchmarks as many tasks aren't fully cross-platform, or some are heavily stilted towards a single platform (example, Java drastically performs better on Linux than Windows or macOS). That said, all things considered, Windows generally performs better than macOS by a noticeable margin, be it better-assisted technologies like CUDA, or DirectX, better ports, and sadly many native OS operations.

    The last great push forward feels like 10.9 Mavericks when Apple introduced Timer coalescing, Memory compression, App Napping, 4k support/Retina for all (10.9.3), maximum RAM increase beyond 96 GB, IPoTB (ThunderBolt transfering between two Thunderbolt equipped Macs), ability call/receive Facetime and a notification system (finally putting growlr to rest). Compare that against Windows 8.1 (both OSes release in 2013) which was almost entirely damage control. I used to confidently say OS X was the best, but in the era of macOS my answer far more pragmatic: "it depends".


    macOS Catalina 10.15 is a disaster

    Every year, Jason Snell of SixColors does a round-up of Apple via a scorecard from various Apple pundits, the interesting takeaway was Catalina was a bad move. I always find this interesting, as an (exceptionally) minor one myself. Gruber has some of the most damning feedback.

    If Apple has somehow determined that typical users need these sort of permission alerts, fine, but there should be a single switch for expert users to toggle to effectively say “I trust all of the software on my Mac”. Call it “Pro Mode”, call it “Developer Mode”, call it “Expert Mode”, whatever. But I don’t know a single expert Mac user who is not seriously annoyed by the heavy-handed security design of Catalina. Not one. Every single expert user I know is annoyed. That is a bad place for MacOS to be. MacOS 10.16 needs a serious course correction to fix this, and if 10.16 goes the opposite way — growing even more heavy-handed in restricting professional Mac users from just using their machines as they want and expect to — I genuinely fear for the future of the Mac as a platform for serious computer users. Which is crazy considering that Apple just unveiled Mac Pro workstation hardware that can cost upwards of $50,000. - John Gruber, My 2019 Apple Report Card, daringfireball.com

    Catalina marks the most significant regression in macOS functionality since Yosemite, with glitches aplomb. I've put off upgrading on both my Mac Pro 2010 and my MacBook Pro 2017 as I just didn't want to deal with the headache. This is only a delaying tactic as inevitably, I will be required to upgrade. While 10.14 was controversial for many users as it switched to Metal, it had to happen as OpenGL was a dead-end, and it made little sense for Apple to put it's GPU fate again into another OpenSource project when it had its own API.

    Catalina's security conscience, as a friend pointed, was a bad pattern when Vista adopted it. It effectively trains a user to suffer alert blindness, and it actually encourages power users to disable SIP and turn off code-sign verification. Apple still refuses to introduce basic detection for when its walled-garden is inevitably scaled. All it can do is revoke certificates. This isn't very good. Having used Windows 10 and tried to use sketchy utilities like a save-file editor for an older video game, I'm impressed how reliably Windows Defender rejects applications. Granted, I'm not much of a Windows 10 user, so perhaps I'm skewed by ancedotalism but fact remains: Apple has no analogous feature to Defender. It still operates primarily on security-through-obscurity and user-hostility to promote its own app store.

    Then to add insult to injury, Apple removed 32-bit Application support. There's plenty of older media utilities that'll be forever sealed to a 32 bit OS, I own a few audio utilities, and there are Steam games I have that aren't Catalina compatible. 32-bit executables weren't holding anyone back. Windows 10 has no plans to phase it out. Catatlina is exceptionally low on meaningful updates, sidecar being one of the big sells but strangely locking out hardware that is still capable of running it.

    The OS has always been the catalyst for all my Apple purchases. I prefer macOS to Windows/Linux, and I prefer iOS to Android. I'm pretty sure any working professional doesn't by Apple products because of their hardware merits alone as I've yet to read of Windows users buying Macs only to run Windows full-time. You can make a strong case for the iPhone and iPad as it's indeed best in class. The Mac? Much less so. Apple is playing a dangerous game with macOS.

    If I were scoring Apple's properties: the Mac line-up would get a C+, as most of its laptops still have goofed keyboards, the prices hikes have been rough, the iMac Pro is stuck. The Mac Pro is beautiful but priced out of reach in a way that feels out-of-touch. For its software, a D. iOS 13 was painful in its first releases and Catalina marks regression in functionality and bugginess. Also, there's been some idling on Logic and Final Cut Pro, although its new services all seem solid (Apple Arcade, News, TV+) and yet it's biggest win, Apple Maps landed in 2020.

    I'm keeping my fingers crossed with Apple's renewed interest in "pro" comes OS upgrades and software worthy of its high priced hardware.


    Nova Beta 7 Impressions

    I'm writing this blog post using Panic Software's newest IDE, Nova.A lot has changed since the first release of Coda in 2007. At that time, we were still primarily making websites, and thus it was an all-in-one-tool tool for a different era where an FTP (yeah, remember those?), a web preview, terminal, and editor all in a single window akin to Dreamweaver without all the bloat. Panic holds a special place in my heart, as they're based here in Oregon, and hailed from PDX long before it was "Portlandia," I was a super fan of Audion, used Transmit for quite a while and as fate would have it, I work two blocks from Panic's downtown location.... I fully admit I change the color of their sign colors every now and again.

    I used Coda from about 2007ish to 2013, and professionally during 2010-2013 and served, forever leaving Dreamweaver behind. I've since gone from Sublime to Atom to WebStorm, but fingers crossed, I'll circle back to the Panic.

    I'm struck deeply by how much the "IDE" application has morphed into its own sub-genre of UX that defies the native OS's visual language. Any of the usual UI is generally disregarded; File trees, scrollbars, icons, gradients are all abandoned. The multi-pane interfaces look nothing like what we've come to expect as the OS standard. In short, code editors seem to exist outside of their host. I have a feeling outfitted with similar color schemes. The average user would have a tough time identifying Visual Studio vs. Atom vs. Sublime vs. Webstorm at a quick glance. However, Nova is certainly the child of Coda, as it carries a similar "Mac-like" feel. It looks fantastic in its black-text-on-white-background. For whatever reason, the dark theme doesn't seem as "nice," and it's largely due to the Apple UX choices. Gradients, transparencies, and San Francisco (the font). It's all here.

    Screenshot of Nova top, Atom middle, Webstorm bottom

    Picture: Nova top, Atom middle, Webstorm bottom

    Here's what I've noticed thus far in a bullet point of unorganized thoughts:

    • My first order was to switch the default font to Operator Mono. I'm happy to see ligatures are fully supported, but it's not making use of italics; thus, the beautiful monospaced cursive I love isn't rendering properly. I don't see any documentation on theming. Maybe I'll look under the hood later...
    • In-window organization feels a bit more natural than the other IDEs, allow you to drag and position windows nicely. However, trying to split both horizontally and vertically doesn't work as it should. Based on the UX, I'm sure it will.
    • It feels zippier than Atom/VisualStudio. Electron kinda sucks and hence why I went to Webstorm.
    • The visual language all about denoting tabs by a top-border. Blue is a local document, and green is a local terminal, and purple is a remote server. The borders are only visible on the front window.
    • Auto tag completion works as expected. Tag/bracket highlighting is nice is and well done.
    • Docker annoyingly triggers the terminal process forever spinning icon.
    • I can't rename terminal windows like Webstorm. It does name the name of the active task.
    • Auto soft-wrap is, by default, on. I'm always curious as to who doesn't use soft-wraps? Am I the unpopular opinion guy when I think this should be the default?
    • Extensions are the main event. Sublime paved the way on this, and Atom/Visual Studio ran with it. It's pretty clear what's the big new killer feature. Pretty much all the documentation that exists currently for Nova is for extensions. It makes sense, functionality for IDEs should be flexible and Nova is no different. The only question is can it attain critical mass? Hopefully.
    • Edit: Tried Nova on my 25,000+ word blog post and the delay between each keypress and rendering to screen was brutal. Looks like Nova for now isn't as speedy as Coda 2 or even Atom.

    All-in-all, Nova feels like progress.


    Diskwarrior: 'Directory must be rebuilt from a recovery disk because of MacOS security restrictions' error and fix

    Diskwarrior still even two decades and change later is a lifesaver. I found my work Time Machine HDD had become corrupted, and Apple's disk utility couldn't fix it. However, I received a "Directory must be rebuilt from a recovery disk because of MacOS security restrictions" error from Diskwarrior. Fear not, you do not need to do this. Under 10.14+, you'll need to grant it the proper permissions.

    Step 1: Open the Security and Privacy panel in System Preferences

    Go to the Security & Privacy Preference pane in the System Preferences, and click the Privacy tab. Scroll Down on the left-hand side to Full Disk Access.

    Step 2: Unlock and click the Plus button

    You'll need to click the Unlock button and enter your password to ungrey out the +/- buttons. Once you've entered your password, click the + button as pictured.

     Privacy tab in Security & Privacy Preference pane in the System Preferences

    Step 3: Locate Diskwarrior and add it

    By default, Diskwarrior will install into the Applications folder, locate it (or where you store your copy of Diskwarrior) and click open.

    The + button action in Privacy tab in Security & Privacy Preference pane in the System Preferences

    Once you've completed this, you should see Diskwarrior in the Full Disk Access list.

    Diskwarrior in Full Disk Access in Privacy tab in Security & Privacy Preference pane in the System Preferences

    Congrats, relaunch Diskwarrior, and you should be able to use Diskwarrior. Fair warning, Time Machine repairs can take hours and a lot of RAM.


    Beats PowerBeats Pro Impressions

    My girlfriend at her company holiday party won a pair of PowerBeats Pro by Beats and gave them to me as she already has the Airpods. I haven't personally owned a set of wireless earbuds ever, as it never been a huge ordeal for me to use wired headphones, and Bluetooth still kind of sucks. They also have the benefit of not requiring a charge, meaning one less thing to charge repeatedly in my life. That said, the place that wireless headphones are attractive is working out, and the beats are designed for such an activity. Here are my collected impressions as someone who's finally slipped into wireless.

    • The beats aren't nearly as comfortable as other ear wrap designs. The early Shure Earbuds, the e2cs used a cable ear wrap design I found more comfortable.
    • The ear hooks are moderately goofy to put on even after a week I'm still a bit cumbersome putting them on or removing them.
    • They rest against the ear instead of in the ear. I bought foam tips hoping to change this. They only minimally block out the outside world, making them not the best for noisy environments like riding a bike to and from work or the gym.
    • Apple has completely screwed over the consumer forcing wireless headphones. I still hate it.
    • Apple knows the Bluetooth experience is shitty. Hence it has developed its own proprietary system with its W1 and now H1 chipsets as opposed to making it an open standard or at least something like MFI where manufacturers can produce there own. The experience of pairing is wonderful for the iPhone and they way wireless headphones should work, but Apple purposely has kept the H1 experience limited to the Airpods line and a few select Beats headphones.
    • The 3D presentation of the beats devices on iOS is garish and feels like an advertisement. If I'm at the point of pairing the device with my iOS device, it's pretty obvious I know what it looks like as it's in my hand.
    • I've only owned one pair of Beats headphones (the original earbuds). I bought 15 years ago for $99 at an Apple store. These aren't nearly as overblown in the soundstage but are certainly bass-heavy, but slightly thin, with an airy/breathy feel. I'd say that audio quality is less than that of 1more Triple drivers, which MSRP for $99 but have tumbled down into the $65-70 range. The PowerBeats Pro's soundstage is simply just ok, with an exaggerated soundstage, and overly bright treble and meaty bass at the expense of rich mids.
    • <liThe charging case is big. The magnet functionality, though, is undeniably cool and handy.</li>
    • The controls work just like a cord on a headphone, although I find myself hitting the play/stop button when inserting into my ears.
    • Auto-pausing when removing an earbud is nice.
    • Call quality is about as good as I've experienced from any type of headphone. It's honestly my preferred method for speaking on the phone over speaker now as there's no risk of cable noise. My girlfriend reported "clicking sounds" like I was typing when I was walking outdoors at the beginning of a call but then said it stopped shortly after. Otherwise, there's been little complaint from anyone I've talked to.
    • With corded earbuds with mic controls, often sweat causes the play/stop to trigger, it is nice that this doesn't happen when working out.
    • I can charge and listen to my iPhone. I'd be able to do this if Apple hadn't actively blocked case makers from producing cases with audio jacks.
    • The looks are in the eye of the beholder. I'm indifferent. The beats logo doesn't mean much to me, and I do like that I'm not wearing the same white earbuds everyone else seems to have. That said, the Airpod Pros seem like a better product.
    • The lack of noise blocking is my biggest point of contention with these.
    • The foam tips are better than the rubberized tips they ship with as they reduce the "thud" effect from walking.
    • Battery life seems pretty good.
    • The range on the PowerBeats Pro is much better than expected, I've walked 50 feet away and still had a connection my large office.
    • The PowerBeats Pro immediately showed up in MacOS as already authorized in MacOS which is a nice touch but there isn't a snappy "hand off" which would have been nice. I don't expect seamless uninterrupted audio, but it'd be nice if I could I could set up auto switching. I'd envision as any not system alert from the iOS device or MacOS device triggers the take-over (assuming the beats aren't already receiving an audio stream). Pause your audio on your iPhone, and hit play on your Mac and the beats auto pairs to the Mac, and then repairs to the iPhone when the Mac is paused and the iOS device plays an audio stream.

    Ultimately, had I paid $200-$250 I would have experienced some buyers remorse. These are not my daily driver earbuds as they're not as comfortable, nor do they block out sound (which is great when doing noisy chores like vacuuming, or biking etc). They're (mostly) a pleasure to use but the fit/comfort is subpar and the lack of noise blocking is surprisingly bad. I imagine its Airpods Pro or bust for the Apple headphones.


    Just a quote...

    “I never understood wind, I know windmills very much, I have studied it better than anybody. I know it is very expensive. They are made in China and Germany mostly, very few made here, almost none, but they are manufactured, tremendous — if you are into this — tremendous fumes and gases are spewing into the atmosphere. You know we have a world, right?”
    - Donald Trump, Dec. 21. 2019

    From the man, who thinks you can nuke a hurricane comes this verbatim quote.


    Setting up Operator Mono in Webstorm (or other Jetbrains/IntelliJ IDEs)

    I noticed there aren't any clear instructions on setting up Operator Mono in Webstorm or any of the Jetbrains IntelliJ IDEs. Changing the font doesn't work quite like one would expect, like the italics, but you can get it working with a little TLC. Out-of-the-box, my copy wasn't supporting alt-characters for italics and it needed to use a fallback font for characters. Webstorm like new IDEs support characters regularly not found in many fonts, called Orthographic ligatures (ligatures). These include less common font glyphs and assist by saving space and are more distinguishable. For examples of ligatures, see Medium.com Ligatures & Coding by Andreas Larsen. Adding the ligatures also seems to fix Operator Mono's italic character set although it could be Operator Mono vs Operator Mono SSM (Screensmart).

    1. Download the latest release from the operator-mono-lig project and decompress it
    2. You'll need to copy Operator Mono into original the directory located in the newly created folder, operator-mono-lig.
    3. Next, open a terminal window and install fontools, pip3 install fonttools
    4. Navigate to your newly created operator-mono-lig folder in the terminal and run npm install
    5. The run ./build.sh from the root directory
    6. Your newly created fonts will be located in the build directory
    7. Next, install your fonts. The easiest way to install the fonts is to select all the fonts in finder and double click, then click install. This will install them correctly and appear as new font family, Operator Mono Sm Lit, or something similar.
    8. Go into Webstorm, Open preferences, Editor, then Font, and select your newly created font.

    To get Webstorm to behave like Atom or VScode, you'll need to manually edit the code styles to use the italics. It only takes a few minutes. I recommend viewing code within Atom as a reference point using Operator Mono as it makes very good use of Mono's italics. I also have guides for Setting up Operator Mono for Atom and Setting up Operator Mono in Coda.

    Troubleshooting: I noticed that the original Operator Mono font I had didn't work, I redownloaded Operator Mono SSM, and it worked. SSM is the limited weight set that removes several weights like thin, ultra). File names cannot have any spaces. I'm also using the material UI for WebStorm, which may or may not make a difference.


    The Mac Pro Buyers Upgrade mini-guide

    I try and not recommend hardware based on my preferences on the Definitive Mac Pro upgrade guide, but here are my opinions for anyone looking for a cheat sheet on what to buy.

    CPU

    Mac Pro 4.1/5.1 configurations vary quite a bit but there's really only two CPUs to really consider due to pricing. The x5680 is cheap, even with a dual CPU Mac Pro, its roughly $70 to purchase 2 CPUs, making it almost the same price as a single x5690. The x5690 is the best CPU a Mac Pro 4.1/5.1 can house.

    GPU

    GPUs on the Mac Pro are limited to the AMD sphere for 10.14 Mojave and 10.15 Catalina. The RX580s are floating around often for cheap, and they're a good entry-level card. The 560 and 570s are even cheaper, but it is hard to beat the price-to-performance. The Vega 56 is probably the best overall value as its performant and can be flashed to a Vega 64 and lands just shy of the Vega 64 in performance after flashing. The Radeon VII is the king and hard to come to buy. At $700, it's expensive but much mightier in the compute benchmarks than the 5700 XT, making it a much more well-rounded card for video editing and other GPU accelerated non-gaming tasks.

    • Good: RX580 $100 (used) or $185 (new)
    • Better: Vega 56 $225 $300 (new)
    • Best: Radeon VII $600 (used) $699 (New) (Mojave and above only)

    Soon the Radeon 5700 XT will be supported for the Mac which occupies a space between the Vega 56 and Radeon VII. The Radeon 5700 XT is great for gaming but it computational scores are low, it performs much lower than the Vega 64 in Blackmagic's Da Vinci Resolve.

    Storage

    There's a lot of storage options for the Mac Pro. The Mac Pro doesn't support bifurcation, so inexpensive dual PCIe cards out a no go. The SanDisk Ultra 3D splits the price and cost intersection nicely besting a lot of the really cheap SSDs like Kingston, although the Samsung 860 is a better SATA drive. Moving the drive to SATA3 doubles the max-transfer speed. The HP EX950 again is another splitting the middle drive between price and performance, in spitting distance of the Samsung Evo 970. NVMe requires a firmware flash for 4,1/5,1s and much more leg work for 3,1s.

    Memory

    Not a lot to say, for the 4,1/5,1 buy 1333 Mhz, go for 16 GB DIMMs if you can afford it.

    I/O

    There's no reason to mess around on the cheaper solutions, Sonnet USB cards are problem-free and do not require external power. The Allego Pro and Allegro 3.1c are the same, featuring two USB 3.1 controllers with 10 Gbps (2.5 GB/s) for 4 ports. The only difference is the interconnect. The regular Allegro has one USB controller for 5 Gbps (1.25 GB/s) total bandwidth.

    Wireless

    Wireless is slightly annoying, but there are three options: Use a PCIe card, use a mini-PCIe card, or buy the parts online separately. Honestly, its best to just read the very long upgrade guide.


    The disrupters will be "disrupted"

    This example is not a hypothetical. The meal-kit company Blue Apron revealed before its public offering that the company was spending about $460 to recruit each new member, despite making less than $400 per customer. From afar, the company looked like a powerhouse. But from a unit-economics standpoint—that is, by looking at the difference between customer value and customer cost—Blue Apron wasn’t a “company” so much as a dual-subsidy stream: first, sponsoring cooks by refusing to raise prices on ingredients to a break-even level; and second, by enriching podcast producers. Little surprise, then, that since Blue Apron went public, the firm’s valuation has crashed by more than 95 percent. - Derek Thompson, The Millennial Urban Lifestyle Is About to Get More Expensive, The Atlantic

    I don't usually do much news commentary but I've been skeptical of the gig-economy and mommy-services for ages.

    The bigger question for me has, how did Silicon Valley start-ups manage to fleece investors for this long? The example that jumps to my mind is MoviePass, which was hemorrhaging money so fast that it went as far as to change user passwords to keep users from using its service. Many of these services existed as a "hack" via legal shenanigans. Labor laws are catching up, as Gavin Newsom signed AB 5.

    When these services actually charge what it costs to use them, like eScooters, or have the double-whammy of having to raise prices to be profitable and content with actually employing the people they're exploiting and be profitable, it's going to be brutal. Many customers will be priced out from food delivery to ride-hailing services. They'll be just like the services they "disrupted" but only with a nicer app to show for it.


    Visual CSS Regression with Backstop JS

    In 2015, I wrote an article, Visual CSS Regression Testing 101 for Front End Developers, where I covered the two competing philosophies of Visual Regression testing, Comparative vs. Baseline. Since then, PhantomCSS was sunsetted as PhantomJS was not as good as running Headless Chrome and BBC's Wraith works but wasn't ever as useful as I'd of liked.

    What is Visual Regression Testing?

    There are other primers on the concept, but it's worth quickly covering visual regression testing. In the course of development, CSS/JS/templating changes can potentially have unintended changes on your website or web app. Visual Regression Testing seeks to automate the laborious task of comparing visual elements to see if any unexpected changes have occurred. This is performed by running scripts with headless web browsers to render the webpage, then capturing its renderings, and using a show diff tool to compare the screenshots, flagging changed elements for review. Once approved, the latest changes are "approved" as the gold master and then saved to compare against next time you run the test.

    Now, four years later, Backstop.JS emerged, mixing (mostly) the best of both Wraith and PhantomCSS.

    Back when I first investigated visual regression, I spent time discussing baseline and comparative tools. Baseline visual regression tools in the talk I attended were complete screen renders, whereas the comparative tools could query individual DOM elements. In hindsight, The distinction between baseline and comparative is somewhat of a moot one, as comparative tools can do baseline checks as they're able to query the screen, be it the entire body. That said, tools like Wraith that only do full-page screen renders can't make individual element selection thus are far more limited. At this point, I doubt either term gets much play, nor does it need the distinction as people have gravitated naturally to a tool that can query DOM elements.

    Backstop.js

    Backstop.JS gets major points out the gate as easy to use. Just run the global npm installer, then navigate to your project directory and run backstop init. It'll create a boilerplate template ready for you to start writing tests. This a serious upgrade, considering I once wrote a 12-step guide on how to install PhantomCSS.

    Running tests is also easy, run backstop test from the root directory and backstop will take care of the rest. Approving a batch of changes is easy, just punch in backstop approve.

    Next up is formatting: All the tests are created in using JSON, which is easy to read and familiar. I've never been super into YAML, and I like JSON. Everyone likes JSON.

    Where Backstop shines is how quick I went from never having written a test to having queried a roster of visual elements found our company website. Start up by declaring a set of screen sizes, and I created my own mobile, tablet, desktop, and large desktop screen sizes.

    {
      "viewports": [
        {
          "label": "phone",
          "width": 320,
          "height": 480
        },
        {
          "label": "tablet",
          "width": 1024,
          "height": 768
        },
        {
          "label": "laptop",
          "width": 1280,
          "height": 800
        },
        {
          "label": "highdef",
          "width": 1920,
          "height": 1080
        }
      ],
    }

    My first tests were entire pages, then I quickly graduated to advanced Backstop, testing our mobile menu. The mobile menu had a few considerations:

    • It must be clicked
    • It only makes sense to test it on a mobile resolution
    • There's a delay for the animation

    {
      "label": "Emerge Menu Open",
      "cookiePath": "backstop_data/engine_scripts/cookies.json",
      "url": "https://dev-site-url",
      "referenceUrl": "",
      "readyEvent": "",
      "readySelector": "",
      "delay": 20,
      "hideSelectors": [],
      "removeSelectors": [],
      "hoverSelector": "",
      "clickSelector": ".hamburger",
      "postInteractionWait": 1000,
      "selectors": ["header #site-navigation"],
      "selectorExpansion": true,
      "expect": 0,
      "misMatchThreshold" : 0.1,
      "requireSameDimensions": true,
      "viewports": [
        {
          "label": "phone",
          "width": 320,
          "height": 480
        }
      ]
    },

    And there you have it; my mobile navigation is being tested against JS breakage and CSS changes. I'm fairly impressed. There's even integration for Running custom scripts. The only hiccups I've had is with AJAX content. I used remove element to hack out the DOM elements, which created reliable elements to test around the AJAX content, and for the AJAX content itself, I used the readySelector.

    Lastly, chaining events is a bit cumbersome as you'll be coding up scenarios, but its still much less overhead than the days of PhantomJS.

    Chaining Backstop to deploys

    The next step is to chain backstop test to deployments. The demo shows Backstop playing with Jenkins deployments. At my office, we use bitbucket pipelines. It's a matter of translations.

    Git flow

    The gif work flow is pretty straight forward with Visual Regression testing, ignore the test folders, and track the gold masters. Backstop creates a new timestamped directory for each test in /backstop_databitmaps_test for each test. Depending on the number of tests, you run, it's easy to churn out hundreds of megabytes of images, so be prepared to have a trash collection method if you're running via a deployment method that might require such.