Longevity of recordable blu-ray discs (BD-R / BD-RE)

▼ So now I got that new optical drive and can start burning CDs, DVDs, BDs (regular 25 and 50 GB blu-ray discs) and BDXLs (newer 100 and 128 GB BDs). But how long will those last?

For each of these types of optical disc, there are three types: mass produced, recordable and rewritable. The mass produced ones are made by stamping physical pits and lands into polycarbonate, cover this with a reflective aluminium layer, and then a lacquer (CD) or polycarbonate (DVD/BD) top layer. DVDs and BDs can have two data layers, BDXLs have three and sometimes four, with semi-transparent reflective layers.

Recordable CDs and DVDs (CD-R, DVD-R and DVD+R) have a layer of photosensitive dye below the reflective layer. The CD/DVD writer uses its laser at higher power to "burn" into the dye layer. With the result that even early CD drives can play CD-Rs without issue. DVD+R improves somewhat on the burn process used by DVD-R.

I'll cover BD-Rs/BDXL-Rs below.

Rewritable or recordable/erasable CDs (CD-RW), DVDs (DVD±RW), BDs (BD-RE) and BDXLs (BDXL-RE) all use a phase change material layer to hold the data. Depending on the intensity of the laser, the phase change material may end up in an amorphous or a crystalline state, which each having a different optical properties.

Longevity: the reflective layer

Obviously all types are sensitive to scratches from poor handling, but the advantage of optical discs over vinyl records and tape is that playing/reading them doesn't involve any contact with the part of the disc that holds the data. So mass produced discs can be played an unlimited number of times without any degradation. For the recordable types, there is a possibility that over time, the reading laser will affect the data layer. But unless you're playing one of these on repeat for months on end, that shouldn't be a problem in practice.

It's widely recommended to store discs in jewel (CD) cases or similar cases where the disc is only held by the hole in the middle, so there is nothing touching it or exerting pressure. And store them vertically.

For mass produced discs, the main issue is that sooner or later, oxygen will reach the aluminium reflective layer, which will then start to corrode and eventually render the disc unreadable. This disc shows lots of small holes in the reflective layer, but so far it plays just fine.

Recordable/rewritable discs often use silver or a silver alloy for their reflective layers. Silver won't corrode immediately just from being exposed to oxygen, but it will in the presence of pollutants like sulfur oxides.

According to Verbatim, a maker of such discs, it's possible to tell the difference between an aluminium and silver blank discs by holding them up to the light and seeing if there's any light coming through the disc. If there is, the reflective layer is silver. If there isn't, it's aluminium.

Of course this doesn't work if there's an opaque layer on top of the disc, for instance, if it's a printable disc. These are pretty nice, by the way:

Longevity: the data layer(s)

The photosensitive organic dye layer in CD-Rs and DVD-Rs is also somewhat of a liability: it will degrade over time. CD-Rs come in various colors that may or may not indicate the type of dye used, with better ones being developed later on.

DVD±Rs always look purple but of course came along when better dyes had been developed, anyway. In practice, I found that out of about a dozen CD-Rs that I burned between 2000 and 2004, one just wouldn't be recognized at all and two others had problems reading a few of their files. For DVD-Rs and DVD+Rs that I burned between about 2003 and 2010, I had issues with about 5% of the discs I tried to read recently.

Rewritables already have the issue that the difference between the two states of the data layer material are much less obvious, but it must also be able to change in two directions, making the addition of room for error much harder. In practice I've had relatively poor results with CD-RWs and DVD-RWs. Sometimes they won't even verify after a burn, and ones from the 2000 - 2010 timeframe had a failure rate of more than 75% when I tried to read them recently.

No problems so far with a box of fresh 25 GB BD-REs, though.

BD-R HTL vs LTH

Which brings us to recordable BDs/BDXLs. Unlike their predecessors, those originally didn't use an organic dye layer to record data, but a composite material, which is more stable. These discs are referred to as high to low (HTL), which gets at the reflexivity of the disc: a blank disc has high reflectivity, which becomes lower when burning.

However, a few years in the disc manufacturers brought the organic dye approach to BD-R, which has the property that the blank disc has low reflectivity, and then gets more reflective after burning, so "low to high" (LTH). This way, the manufacturers could reuse their existing CD-R/DVD+R production lines with only small modifications. HTL discs, on the other hand, require a more complex sputtering production method.

The way I see it, the only reason to still use optical for storage is because of its longevity. The good part is that you can replace a drive and then still read all your existing discs. With hard disk drives and flash, once one part fails (mechanics? electronics? the data?) then you're pretty much left without any useful course of action. In that light, the adoption of LTH BD-R is a step back.

A Reddit contributor felt much the same, but also concluded that LTH BD-Rs had completely replaced HTL BD-Rs in 2018. (Worth reading just for the first comment: "This is one of those posts that shows that it's entirely possible to be 100% correct, and also 100% bat-shit crazy.")

Fortunately, things don't seem to be quite this bad. Yes, it's annoying and perplexing that disc manufacturers don't label their products with important information like whether the discs are HTL or LTH. But, it does seem HTL BD-Rs are still being made, as I ordered a spindle of 10 25 GB Verbatim 6x speed inkjet-printable BD-Rs before I did any research... and it looks like those are HTL. Using Apple's System Profiler, we can see the following under Disc Burning:

We can then look up the ID over at the Disc Manufacturer ID & Media Type ID Licensee List and find out that the CMCMAG BA5 disc type is actually HTL.

Also, if you're careful (it's easy to get confused), you can tell by looking at a burned disc whether it's HTL or LTH. The two discs at the top are a blank DVD+R and a used DVD-R. The ones at the bottom a blank BD-R and a used BD-R.

If you look carefully, you can see that the unused part along the edge on the top right DVD-R is slightly darker than the burned part. For the BD-R at the bottom, it's the other way around, which is easiest to see around the middle. The first ring is very dark, but that one isn't writable. The next ring is writable material, but outside the writable area. This doesn't show up all that well on photos, and it's also easy to misinterpret. But there is a visible difference.

M-DISC

M-DISCs are supposed to last for 1000 years. The accelerated aging tests aren't proving them wrong, but of course also not proving them right. (Wait another 990 years for that.) But at least M-DISCs are making a claim about longevity, which is more than the competition does.

What to do?

If you have data that you want to stay around for a long time, without having to copy it from one HDD or SSD to another every few years, optical is probably the best choice. Still, just writing your data to disc once and then assume it will remain readable for decades is too optimistic. So I'd say:

• If possible, check the error levels for every disc you burn. Unfortunately, this information is generally not available.
• Store everything at least twice, preferably three times. Apply optical disc storage best practices.
• Use different types of discs and/or storage for the different copies.
• Also try to keep the different copies in different locations, both to accommodate for large scale disasters such as fire and floods, but also more mundane stuff such as less than ideal temperature and relative humidity.
• Check your discs once every decade or so, and make fresh copies when in doubt.
• Single layer (25 GB BD) is probably a safe bet, with less to go wrong compared to multiple layers.
• BD-R M-DISC is probably best, with BD-R HTL in second place. Whether BD-RE is better than BD-R LTH is unclear. So when in doubt, use both.

Permalink - posted 2022-01-09

Optical storage and media in the '20s

▼ Last year I wrote posts about listening to the radio, watching TV and having a landline phone in the 2020s (in Dutch). Today, I want to talk about the place left for optical storage in the 2020s.

Do I actually want or need those shiny optical discs in my life?

The short answer: most people probably don't. Software is now always available online, so you don't need an optical drive to install applications or drivers. For music, you can get pretty much everything through streaming services such as Spotify. And if you bought CDs back in the 1990s, you'll certainly have ripped those by now. I think you can watch pretty much any movie on one online service or another. Current TV shows are also generally available on a streaming service.

Transitioning to the disc-free lifestyle

But what if you bought movies and TV shows on DVD in the past, and used to burn some of your data to CDs and/or DVDs? You really want to go over all your data CDs and DVDs and copy those to a harddrive while you still have a working optical drive. That optical drive will die at some point (or become unreliable to the point of being unusable). You can of course then spend less than $/€ 50 for a DVD drive and handle this chore then, but why wait? Those CD-R and DVD±R organic dye layers aren't getting any better as they age.

Ripping movies: meh

I'm not sold on the option to rip all my movies and TV shows on DVD and blu-ray to my NAS or an external HDD. That's just way too much data. For movies, I don't have too much of an issue with simply using my blu-ray player (the one connected to my TV) to watch them. Yes, finding the right disc and then waiting for the drive to do its thing and having to deal with unskippable warnings and annoying menus isn't great, but that's only a minute or so vs a movie that's at least 90 minutes.

Ripping TV shows the hard way

For TV shows, this is quite different. Especially when I want to watch a 22 minute sitcom episode (and boy, do I love those turn-of-the-millennium sitcoms!), having to wait for the player to do its thing and then the warning and menus and me forgetting which episode was next, it's all quite annoying.

So what I used to do before my the Apple USB Superdrive that I got in 2011 kinda-sorta died recently, was rip TV shows from DVD and then use Handbrake to convert them to H.264 format, put in all the iTunes metadata using Subler, and store those on my NAS. However, that is quite a lot of work and eats up quite a bit of storage.

Ripping TV shows the easier way

Then I found an easier way to handle TV shows on DVD (and, to some degree, blu-ray): MakeMKV. MakeMKV is an app that, well, makes MKV files. You can use it to rip DVDs (easy), regular blu-ray (a bit more difficult) and UHD/4K blu-ray (when the stars align). My new blu-ray burner mentioned below didn't give MakeMKV any trouble when ripping DVDs, even region 1 ones on this region 2 drive.

For DVDs with TV shows, it's relatively straightforward to select the right tracks and turn those into .mkv files. However, MakeMKV doesn't do any transcoding. So the video is still MPEG 2, and the audio is still the same (usually) AC3. This means the files will be unplayable in many places, including iTunes and the iPhone/iPad.

However, my LG TV is more than happy to play these files. And, being a TV, it handles the intricacies of framerates and deinterlacing well automatically. fSo I just rip one DVD at a time, put the MKV files on a USB flash drive, and stick that in one of the USB ports of my TV. Yes, the files are three times as large as compared to the handcrafted H.264 versions, but I don't have to think about any settings and just delete the files after watching. Should I want to rewatch the same show a few years from now, I can always rip the same DVDs again.

My new optical drive

Like I said, my Apple USB Superdrive is no longer working properly, and the idea of having a big box full of music CDs and a smaller box full of data CDs and DVDs that I can no longer read bothers me.

So I decided to get a new external USB optical drive. One that can read and write blu-ray discs in addition to CDs and DVDs, of course. I also had a look at UHD blu-ray, which is used for 4K content on blu-ray. However, reading those requires firmware modifications, if possible at all. Dive into the MakeMKV forums to figure that out.

So I ended up ordering a Hitachi-LG BP55EB40 "Slim Portable Blu-ray WRITER" for a little under € 100. Optical drives used to have some issues with the amount of power provided by the USB2 bus in some cases, but that doesn't seem to be a problem anymore. You'd think USB3 would solve that, but no, as the datarates required for 6x speed BD, 4x speed BDXL (100/128 GB blu-ray), 8 x DVD and 24 x CD reading and writing fall well below what USB 2.0 can handle, that's what we get. The drive pretty much handles all the formats:

It will also write M-DISC discs. Those are recordable DVDs or BDs that use a special material for the recordable data layer(s) that should make them last for a thousand years.

Blu-ray for archiving

I'm not sure if my data still needs to be available in the year 3022, and it probably won't be, even on an M-DISC, but given that my failure rate from the DVD-Rs that I burned between about 2003 and 2013 is quite low (maybe 1 in 25 is no longer readable), an improved type of recordable disc seems like a good bet for things that need to last a few decades.

The advantage of an optical disc as an archival medium is that it only includes the data, not any electronics or logic. It's also fairly small and especially flat, so you can mail them or store them in a folder or safe deposit box and not worry about shocks or drops. A couple of years ago, I wrote about using flash for archiving, purely based on price. However, as flash capacities have gotten larger, especially for USB thumb drives and SD cards, their reliability has dropped. This is not a technology appropriate for long time archival storage.

Also, unlike storing data on a (possibly archival) hard disk drive (HDD), burning a disc is a one-time thing, so it encourages careful consideration of what needs to be archived in what format.

Conclusions

Although I think it's possible to leave the optical past behind, I do think keeping an optical drive around to rip TV shows and burn archival copies of my data is a good idea.

Permalink - posted 2022-01-02

Resizing photos in PHP with correct orientation/rotation and retaining EXIF data

▼ When I got my first iPhone back in 2008, I hacked together a quick script that let me post photos from my iPhone to my website. However, iPhone photos are now 3 MB or larger, which is a bit much. So in practice I would edit the photos on my computer before posting them. But that gets old, so time for some automation.

The first photo I uploaded directly from my iPhone 3G, showing part of the script

Actually resizing images is easy enough in PHP using the imagecopyresampled() function. However, using the provided example script the result was that a photo that I took with my iPhone showed up rotated.

The reason for that is that phone cameras store the camera's rotation in the EXIF data, and the PHP image manipulation functions strip away the EXIF data. One way to solve this is to read the EXIF data using exif_read_data() and then rotate the image accordingly using the imagerotate() function as explained here. However, that still leaves an image with no EXIF data.

A better way to do that is by copying all the EXIF data from the original image to the resized image. This requires a bit of work, but there is a good example transferIptcExif2File() function in the user contributed notes for the PHP iptcembed() function.

Searching for this using Google didn't give me any results, so hopefully the above will be helpful to others running into the same problem. And as you can see, it works:

Permalink - posted 2021-11-28

Test post preparing for HTTPS

▼ For some time now there's been a push to move the entire web from HTTP (no encryption) to HTTPS (with encryption).

I never felt it was necessary to encrypt static content. But the time has come to cave to peer pressure, and enjoy a few benefits such as the ability to post images to web forums.

However, I still feel very strongly about everything remaining accessible over HTTP. If you go HTTPS-only, you are now relying on a certificate authority (as well as your own ability to keep certificates up to date!), and you're cutting off some older systems and losing some performance. Also, having to change everything wholesale will be somewhat of a nightmare.

The trouble is: how do use HTTP images and links on an HTTP page and HTTPS images and links on the HTTPS version of the same page? With dynamically generated pages that's not too difficult: just let the script insert HTTP or HTTPS as required. Or, even easier: use relative links, that don't include the domain name.

Unfortunately, I have my content spread around a number of subdomains, and most of it is in static pages.

Turns out that you don't have to choose between:

http://www.iljitsch.com/2021/11-20-test-post.html (absolute URL)

and

/2021/11-20-test-post.html (relative URL)

but you can also use:

//www.iljitsch.com/2021/11-20-test-post.html

That's a protocol-relative URL. So you do specify the domain name, but not HTTP or HTTPS (or any other protocol). So if you use such a URL as a link on an HTTP page, the link will use HTTP. If you use it on an HTTPS page, the link will use HTTPS. So that's perfect for links within the site and images on static pages that may be served over either HTTP or HTTPS.

Paul Irish has a blog post that initially explained that it's a good idea to use protocol-relative URLs, but the current wisdom (added to that post) is they are better avoided, as there may be some cross site scripting possibilities. But as I don't use any cross site scripting on iljitsch.com that's not an issue here.

So I still need to go over the entire website to make sure protocol-relative URLs are used where I can't use regular relative URLs.

Apart from that, the main complication is my RSS feeds. Unless I'm misremembering, I started using absolute paths for my images because otherwise the images wouldn't show up properly on RSS aggregators. So I probably need to rewrite relative and protocol-relative URLs in my RSS feed script.

This is a test post to see how relative and protocol-relative images and links are handled in RSS. So don't be alarmed if links or images in this post don't work if you're viewing this post through an RSS aggregator.

Let me plug Esperanto and do the opposite for nuclear energy with this relative image (link):

Then again, fossil fuels aren't ideal either, using a protocol-relative image URL (link):

More later.

Permalink - posted 2021-11-20

Review: MouSTer: making my favorite wireless mouse work with my Amiga

▼ Original Amiga mice were never that great, and a few decades of use and storage didn't help. So if there's one part of the Amiga I have no problem replacing with something new, it's the mouse. (Don't feel bad, I swore of Apple mice, too.) So a while ago I got a USB-to-DB9 mouse adapter. That one worked fine with wired mice, which was a big improvement.

An Amiga 3000 "tank mouse", an Amiga 1200 mouse and a 2000-era Logitech Mini Wheel Mouse

However, a few years ago I discovered the Logitech Anywhere MX 2 wireless mouse. I liked it so much I got one for at home and one for at work. The MX 2 is a bluetooth mouse that can pair with three computers. You can simply switch to another computer with a small button at the bottom. Now obviously the Amiga doesn't do bluetooth, but the MX 2 comes with a Logitech unifying receiver, which you can plug in any USB port and then the computer thinks it's talking to a wired USB mouse.

Problem solved, right? Not really. The MX 2 didn't work with that adapter, nor with another one. Eventually I went as far as to get a RapidRoad USB adapter for my Amiga 1200. This does work very well most of the time, but the complex USB software is not compatible with most games and is not 100% reliable.

The MouSTer connected to a DB9 extension cable, with the Logitech unifying receiver and the MX Anywhere 2

So when I read about the MouSTer, I thought four times might be a charm. After waiting for the little device to make its way from Poland to Holland over the course of very many days, it arrived today. So I plugged it in and... not much. The mouse pointer did move a pixel or two, but nothing close to working correctly. Then I noticed:

So I plugged a USB thumb drive into the MouSTer, which produced furiously blinking LEDs. Afterwards, there was a new directory on the flash drive, with in it a file named MOUSTER.INI. This is a text file with settings and clear explanations what they do. Out of the box, the MouSTer was set up for the Atari ST (hm, maybe the capitalized ST in the name should have been a clue?). I quickly changed that to "amiga". For good measure, I downloaded the latest firmware and put that next to the MOUSTER.INI file and put the thumb drive back into the MouSTer. After that, different mice worked as expected, with just some tweaks to the DPI divider and LED heartbeat settings needed.

A modern optical mouse like the MX Anywhere 2 sends way more "hey, the mouse has moved" pulses than the Amiga expects. As a result, when you move the mouse quickly, the mouse pointer jumps around because the computer can't keep up. Using a DPI divider ("microstep") value of 2 - 5 on the MouSTer didn't completely solve that for me, but using the Mac Logitech Options program to tell the mouse to use the minimum pointer speed, along with a microstep value of 2 made everything work satisfactorily.

Amiga mice normally have two buttons. The DB9 port has just one unused wire, so the Amiga can recognize a third button, but that's it. No scroll wheels—without using different hardware. However, the MouSTer will helpfully map any additional mouse buttons to the three the Amiga can recognize, with or without autofire. A program like FreeWheel will let you do something useful with the extra mouse button.

You can tell the MouSTer is starting up after turning on the computer because its yellow-orange light flashes furiously. (It'll tell you any error messages in morse code.) This is fast enough that if you press both mouse buttons as you turn on the computer, you're greeted by the boot menu as intended.

Conclusions: after only a few hours of use, I'm already a fan of the MouSTer. The hardware is nice and small, so it doesn't stick out far enough to get in the way. The electronics are visible, but covered with plastic so no chance of accidental shorts. The configuration method is unorthodox, but works well. I tested it with both a wired and a wireless mouse, as well as a couple of joysticks. Everything worked. (It also supports gamepads.) Using the Amiga mouse port is a mixed blessing. On the one hand, it's a limitation: only the one extra middle button and no scroll wheel support. On the other hand, it works with everything. In my case, I simply use two Logitech receivers: one on the MouSTer and one on the RapidRoad. So I just flip the mouse and push a button to switch between features and compatibility. 😎

Permalink - posted 2021-07-24