|_|_|_|_|_|_|_|_|Intro|_|_|_|_|_|_|_|_|

What a fun week! Phillip's lessons were very informative, and everyone's interesting projects were very impressive and inspirational. I do a little research in school on lossy audio compression algorithms, image forensics, and computer modelling of circuit bent instruments, and also write some music that draws inspiration from glitches and hardware limitations (hear: a recent album, and a recent multi-channel fixed media piece ) and I found that getting more experience working on visual glitch art has really deepend my appreciation and understanding of these subjects. Thanks, Phillip (et al.)!

During this project, I got very interested in writing software to assist with databending, and started spending a lot more time on that than some of the later deliverables. I developed a few methods for using code to assist in data bending, using Matlab (although I also used Synalyze It! Pro to help understand things as I went, it is only used for applying these methods by hand).

The first method is for JPEGs (it works for MP3s too, incidentally, though that is outside the scope of this class). After developing this method, I got very interested in the problem of PNGs. The second and more complicated code-assisted method automates PNG bending, but also helps repair the broken CRC32 checksums that result from PNG tampering, which normally make PNGs very difficult or impossible to bend.

Below, you can find an outline of the methods I worked on, the examples I produced to demonstrate them, some thoughts on my process, a bunch of links to the online resources I used, and the specific deliverables for the class.

|_|_|_|_|_|_|_|_|Outline|_|_|_|_|_|_|_|_|

I. Code-Assisted Glitching
A. Idea
B. Requirements
1. Read/Edit/Write raw data
2. Catch Errors?
B. Pseudo-code example
1. STUTTER QUILT - code-assisted JPEG bend
C. Another example, PNGs
1. PNG STUDY IA - code-assisted PNG bend
2. PNG STUDY IB - code-assisted PNG bend
3. PNG STUDY 1C- code-assisted PNG bend
4. PNG STUDY 2- code-assisted PNG bend
5. SCROLLLL - code-assisted PNG bend
D. More PNGs, header bending
1. CONCEPT PROOF - code-assisted PNG bend
2. SPELUNKING - code-assisted PNG bend
II. Deliverables
A. Lesson 1: "Dozens of Glitches"
1. GLITCH SUNSET - JPEG bend
3. BIRD BENT - GIF bend
B. Lesson 2: "Glitch GIFs!"
1. GLITCH MOUNTAIN - PNG bend
2. PNG DANCE - PNG bend
3. SECOND DANCE - PNG bend
C. Lesson 3: "Audacity Databent Files"
1. Filtering
a. Lowpass sweep
a. Highpass sweep
a. Bandpass sweep
2. BLOOM - bandpass filter sweep on raw TIFF data
3. PHASE100 - phaser on raw TIFF data
D. Lesson 4: "Datamosh Some Images"
E. Lesson 5: "Busted Audio"

|_|_|_|_|_|_|_|_|Code-Assisted Glitching|_|_|_|_|_|_|_|_|

I am experimenting with applying automated methods to databending (and glitching in general). Knowledge of the data structure inherent to various file formats can point to a method for code-assisted/automated glitching. Code that can help with glitching needs to be able to read and write raw data and maniupulated this raw data. Ideally, the code should also be able to catch when a glitch will render a file completely unreadable, and reverse the glitch. I am using Matlab (commercial software), but it seems that Octave (like Matlab, but free and freely redistributable) or Processing (free / open source) could be some good alternatives.

Here is pseudocode I came up with for a code-assisted glitching architecture:

inputFile = 'somefile.jpg'; % the input file name
outputFile = 'somefileGlitched.jpg'; % the input file name
binaryData = fileread(inputFile); % convert it to binary data
for i = 1 to howeverManyGlitchesYouWant % edit the binary data many times
safeBinaryData = binraryData; % keep track of the "safe" binary data in case of bad glitch
binaryData = someFunctionToMessWithIt(binaryData); % glitch it
try
filewrite(binaryData, outputFile); % write it to an output file
testData = fileread(outputFile); % try to read the newly glitched file it
catch % if the image read throws an error
binaryData = safeBinaryData; % restore the old "safe" binary data
i = i -1; % this iteration didn't count, since it didn't produce a usable glitch
end % end of the try/catch block
end

The try/catch block tries to execute the code in the try block, and if it throws an error, goes on to execute ;the code in the catch block. The second line in the try block ("testData = fileread(outputFile);") wil throw an error if the data that had just been written is not a valid file (if it has been glitched too hard). This method is very powerful and fast, and allows you to programmatically sequence glitch generation ("someFunctionToMessWithIt()" could be anything!). However, it is subject to your coding environment's opinion on what is "too damaged." For instance, the case where Matlab considers a particular JPEG unopenable, but Finder or GIMP has no problem with it, is not hard to imagine.

Try/Catch is Matlab's version of exception handling. I haven't dealt with exception handling in many other languages, but it looks like Octave and Processing do have their own versions [Catching Errors in Octave, Exceptions in Processing].

Here's a GIF that results from using this method to databend a JPEG file. A large file is databent by grabbing a chunk of data at the beginning of the file and repeating it many many times (overwriting existing data). Each frame number corresponds to the size of this chunk (frame #1 has a repeated chunk that is one byte long, frame #2 has a repeated chunk that is two bytes long, &c.). The GIF was created by grabbing only a small (256x256) square portion of each image (making GIFs in Matlab is pretty easy).

STUTTER QUILT
The result of early experiments in code-assisted glitching of JPEGs. Code-Assisted Glitching - image 1 - student project

Here is a "quilt" of the first 25 frames of the GIF, if you prefer your images to be non-moving: Code-Assisted Glitching - image 2 - student project

I've been wanting to figure out why PNGs are so difficult to bend. So, I spent a lot of time reading up on the PNG Home Site. The history of the PNG format is actually very interesting, I would definitely recommend taking some time to read about it. I found our how the format protects against errors (it is the CRC32 flavor of Cyclic Redundancy Check). This protection against errors makes it VERY difficult to happen upon stable bends in a PNG (you have to get lucky that your edit happens to create a valid checksum, and since checksums are 8bit numbers... it is kindof unlikely).

The structure of PNG files (outside of the header and footer) is:
(length of data block) + (IDAT) + (a bunch of data) + (CRC32 Checksum)

In hexadecimal, the IDAT marker is x49444154
the length of data marker is also 8 bytes long.

the CRC32 checksum is calculated like this.

So, if you edit some of the data between the IDAT ("image data") marker and the Checksum, it will likely invalidate the file, since the calculated checksum won't match the checksum that is stored in the 8 bits following the chunk of data. So, all you need to do to make the file valid again is to calculate this 8 bit checksum yourself, and replace the existing checksum. When the decoder checks to see if everything is allright, the calculated checksum will match the recorded checksum, and you will get to see some awesome, and previously difficult to catch, PNG bends!

You can use this method to bend PNGs by hand (a tool like Synalyze It! Pro would make this easier). Or, you could do it automatically, similar to the methods I describe above.

PNG STUDY 1A
I bent several very large sections of a PNG file (recall that previously, even small edits on PNGs made them very difficult to work with), using the method described above. Each bend worked on the first time, none of them produced bends that made the file unopenable. Here is a crop of one. The Bayer Filter-looking bit on the bottom came from copying the word "love" thousand of times into the file (d'awwwww). The blue and red reminds me of the SkillShare logo too...
Code-Assisted Glitching - image 3 - student project

PNG STUDY 1B
Here is another crop. These atually end up looking a lot like my other PNG bends (see below), but they are soooooo much easier to make.
Code-Assisted Glitching - image 4 - student project

PNG STUDY 1C
This one ended up looking like a differently-colored version of PNG STUDY 1.

PNG STUDY 2
Another nice bend found using this PNG Checksum recalculation method. A wuite different effect from the others above! Code-Assisted Glitching - image 6 - student project

I found out that you can edit other aspects of PNGs that are normally protected by checksums (for instance, interlace method and color type), not just the image data in between IDAT blocks. For information on PNG chunks, look here. In addition to only picking valid values (most of the possible values are either not meaningless, or reserved but unimplemented), you must repair the checksum CRC32 chunk that comes afterwards (again, this calculates the checksum starting at the type chunk, and ending right before the checksum chunk, excluding the length chunk).

CONCEPT PROOF
I took a PNG of one of my drawings of a chip, and used this method to change the color type from type 6(each pixel is an R,G,B triple, followed by an alpha sample) to type 4 (each pixel is a grayscale sample, followed by an alpha sample). Then, I made two versions, one with interlacing method 00 (the original) and one with interlacing method 01 (Adam7 interlacing), and animated them into a two-frame GIF. This isn't the prettiest, though it shows the concept working. Actually, if you zoom in it isn't bad. Plus, you can see the original chip drawing very faintly.
Code-Assisted Glitching - image 7 - student project

SPELUNKING
Here's an example that is a little cooler. I took one of the PNGs with a bent header and copied a bunch of chunks of data around inside (as well as repeated hidden messages), repaired all of the checksums, and cropped the result.
Code-Assisted Glitching - image 8 - student project

Here's the original chip drawing that the previous two examples came from:
Code-Assisted Glitching - image 9 - student project

SCROLLLL
Another GIF of progressive PNG bending (just changing one byte, and repairing CRC each time):
Code-Assisted Glitching - image 10 - student project

|_|_|_|_|_|_|_|_|Lesson 1 - Dozens of Glitches|_|_|_|_|_|_|_|_|

GLITCH SUNSET
Warm-looking GIMP crop of a larger character-wise databent source (JPEG). Code-Assisted Glitching - image 11 - student project

The uncropped databent image, which came from databending a picture of an old friend/bandmate (in the hex editor):
Code-Assisted Glitching - image 12 - student project

BIRD BENT
A databend on a (static) GIF of the above image (the album cover to The House You're Living In, and album by my old band, World's First Flying Machine).
Code-Assisted Glitching - image 14 - student project

|_|_|_|_|_|_|_|_|Lesson 2 - Glitch GIFs!|_|_|_|_|_|_|_|_|

GLITCH MOUNTAIN
Another PNG bend, 16 frames progressively bent in a hex editor (Hex Fiend), and cropped.
Code-Assisted Glitching - image 15 - student project

This was originally a picture of Mount Shasta, and again I had the experience that the bend only affected a small slice of the image. Matlab throws two types of errors when I try to read corrupted PNG files (which makes it a little difficult to use code-assisted methods), "Warning: PNG library warning: iCCP: CRC error" and "Warning: PNG library warning: Ignoring bad adaptive filter type." CRC means Cyclic Redundancy Check, and my intuition is that this mgiht be part of why I am having a hard time getting large regions of PNGs to glitch out. The Wikipedia article on Cyclic Redundancy Check states that PNGs use (typically) a type of check called CRC-32. So, it seems that understanding this error checking will be key to unlocking easier PNG bends. Perhaps blocks of PNG data can be rechecked, and additional bits altered so that the same error check is generated? Perhaps the error check can be disabled by changing the code (is it transmitted along with the data?) to something trivial?

PNG DANCE
crop of a data-bent PNG. It seems that when I bend PNGs, I end up with lots of cool stuff at the bottom of the image, but nowhere else. Weird? Hey, this got posted up on glitchgifs!
Code-Assisted Glitching - image 16 - student project

Here's one uncropped frame of a bent PNG, showing the weird stuff that shows up at the bottom, but nowhere else (it also gives a good idea of what the source material looks like):
Code-Assisted Glitching - image 17 - student project

SECOND DANCE
The same method applied to another image. This time I also messed with alpha/transparency when assembling the GIF.
Code-Assisted Glitching - image 18 - student project

|_|_|_|_|_|_|_|_|Lesson 3 - Audacity Databent Files!|_|_|_|_|_|_|_|_|

I wanted to focus on one particular class of audio effects that could be applied to raw TIFF image data in Audacity - filtering. I quickly came up with one shortcut for making multiple effected versions of the same image. It helps to use "Split New" in Audacity to move the header and leading TIFF tag to separated tracks. Once this si done, it is very easy to select all of the image data to apply an effect too. If you want to only affect a certain region of the image, split it to a new track as well.

The first thing I tried was different lowpass filter settings. I wanted to reproduce a classic electronic music gesture, the filter sweep, in a databent regime. To this end, I used various cutoff frequencies (by octaves from 20 Hz to 20480 Hz... or 20, 40, 80, 160, 320, 640, 1280, 2560, 5120, 10240, and 20480 Hz), always with a resonance of 12dB. The following four images all came from a photo of a patch bay in the University of Illinois Experimental Music Studios, where I used to do some electro-acoustic composition. Here is the result:
Code-Assisted Glitching - image 19 - student project

Next, I applied the same methodology to a highpass filter sweep:
Code-Assisted Glitching - image 20 - student project

Finally, I applied this methodology to a bandpass filter sweep (Apple's built-in bandpass filter has a bandwidth definition, rather than a resonance/Q definition, so I used a standard 600 cents):
Code-Assisted Glitching - image 21 - student project

BLOOM
My favorite two frames from all of these experiments were the last two frames of the bandpass filter sweep. So, I made a more finely-grained (by semitones, rather than by octaves) sweep between the two cutoff frequencies associated with those frames, and cropped the result to my favorite portion on the image (100x100 pixels from the upper left). I really like how this one exposes the interaction between the layers in the TIFF encoding. Hey, this got posted on glitchgifs, too!
Code-Assisted Glitching - image 22 - student project

PHASE100
Not too much to say about this one, it is a crop of the image that results when high-feedback phasing is applied to raw TIFF data (it's not even animated):
Code-Assisted Glitching - image 23 - student project

|_|_|_|_|_|_|_|_|Lesson 4 - Datamosh Some Images|_|_|_|_|_|_|_|_|

|_|_|_|_|_|_|_|_|Lesson 5 - Busted Audio|_|_|_|_|_|_|_|_|

Kurt Werner

PhD (CBMTA) student at CCRMA

10

2