000webhost

Web hosting

Tuesday, July 24, 2012

The Hard Drive Resurrection!

My first hard drive 'incident' occurred years ago when a friend moved a hard drive of mine while it was still working. The result was 'smoky' and resulted in a drive that would not spin up and a fried controller board. Switching boards for data recovery was simple though as the drive was of desktop size and the model fairly common. 

Since then I've sufferred all sorts of drive failures from virtually all drive manufacturers. While I haven't lost much data (I backup regularly) it has been annoying and I've learnt about hard drive technology along the way. Last year I had another drive failure. This time it was a laptop sized drive with a supposedly 'seized spindle'. I tried 'freezing it' in a sealed container and it seemed to spin up long enough for me to recover the data but after one power up that was basically all the activity that I ever saw again from this particular drive. Since I recovered the data I didn't bother with further experimentation. Recently though, some other work in data recovery brought this drive back into my mind. I opened up the drive and was reminded that even though hard drive technology has been around for decades it has definitely evolved/become more refined. Drive designs from a decade ago there were much more distinctive between drive manufacturers than now and there were sometimes aspects of drive design which could have easily been improved (such as shock absorption and chip placement). Over time, its become clearer that drives have become much more refined though and manufacturers have converged upon some similar design principles. When I opened this particular drive it was obvious how much thought was put into it; there were stickers to prevent improper screw placement (some screws could have protruded directly onto the underside of the drive platter if you weren't careful), contingencies were made to deal with the possible build up of internal condensation/contamination (packet of what appeared to be silica/cotton?), and even minor design changes to head parking areas to reduce the shock of drive movement to the extreme edges of the disc platter themselves.

Anyhow, it seems clear that there are several methods when dealing with seized spindles. One involves using a power drill, another blunt force, another involves a specialised tool to help re-align the platter/bearing bundle. I decided to use a stylus from my old Palm Vx. It seemed to work well and I was at least able to get my drive spinning relatively freely again. The next problem that I faced was head/servo movement. I checked my re-assembly several times and everything seemed to be in check. However, after careful examination I realised that a cable that led from the controller board to the head was deformed/broken which meant that power could not be sent to an electromagnet at the end of the head assembly which is ultimately used to control head alignment over the disc platters. After re-alignment head movement seemed to be back. While its clear what the problem is, I'm not sure how much further I can push this with my existing equipment. The connections/wires that need to be fixed are basically the size of pin heads. It will take a microscope and a bit of luck to be able to accurately re-connect the relevant connections.

Moreover, unlike previously where switching parts seemed to be quite simple I'm discovering that drive recovery nowadays is far more evolved than it used to be. While some parts are interchageable others aren't so easy. Based on some research it seems as though some drives may have begun to place part or all of the firmware on the actual drive itself which makes parts transfer not so simple. The firmware from the old drive needs to be extracted or procured from another location (can be trivial or extremely difficult depending on the situation/manufacturer as some publish tools/software to do precisely this while others rarely publish any supporting documentation/software at all).

I suspect the best/simplest method may be sourcing another drive for its head assembly (if you don't have specialised head removal tools you're best choice is turning the drive upside down and pushing it towards the back wall of the drive enclosure to remove it and enhance your chances of removing it without causing significant drive damage)(Obviously, you've discovered by now that you can often get by without having to resort to specialised tools a lot of the time. However, like general 'tradesman' it is far easier if you have them and you need to take extra care if you don't have them.). I'd like to know whether they alter drive heads based on drive density though? Clearly, when switching from single to multi-platter configurations this will be the case. How about switching from a single platter discs of different density though? Surely, efficiencies in mass production should mean that you use the same heads while altering only the platters, firmware, controller card, etc?

As an aside I've noticed a lot of good deals for SSDs of late. However, it's also clear that some of the drives on offer have a history of data loss as a result of dodgy controllers, firmware, etc... For the moment, I would do my homework before purchasing an SSD, hold off on the technology while it matures, and/or just use it for non-critical/read only purposes.

A word of note, if you're working with this type of equipment you should primarily be thinking about the most minimal way to get things going again rather than seeking the most elegant solution out there (at least initially). I recall working an phone that had sufferred from water damage. Everything seemed to be fine but the power switch needed to be replaced. While I managed to find a replacement from Jaycar I tried to get too cute with the repair and ended up damaging the replacement switch. Luckily, the switch was only 95 cents and the phone was relatively inexpensive.

http://en.wikipedia.org/wiki/Hard_disk_drive

http://www.overclockers.com/do-it-yourself-hard-disk-repair/
http://superuser.com/questions/44318/how-do-i-remove-a-mybooks-wd-smartware-virtual-cd-from-my-desktop
http://club.myce.com/f138/access-hard-drives-firmware-257428/#.UAoich-eVG0
http://freemyipod.org/wiki/Extracting_firmware

- as usual thanks to all of the individuals and groups who purchase and use my goods and services
http://sites.google.com/site/dtbnguyen/
http://dtbnguyen.blogspot.com.au/
at
Labels: , , , , ,

The World is Full of Software Bugs

It seems as though bugs may have developed an attraction to me of late. I've discovered that 'pcmanfm' doesn't handle unmounting of drives completely cleanly and that you need to manually kill processes regularly sometimes. I've also discovered that Windows Media Player (with my configuration at least) only seems to work when the TV tuner is inserted on the original USB port that it was configured on (else it will indicate that there are no tuners currently available. Obviously, you can attempt to re-configure the tuner on another USB port but that's beside the point. This is strange behaviour.). MultibootUSB needs 32-bit of 7-Zip to be installed to be run (even if you have installed the 64-bit version on a 64-bit Windows system). Otherwise, you end up with a never ending series of errors. Tip, you need to kill MultibootUSB first before killing 7-Zip else it will keep on trying to re-spawn that process.


While the following is not a bug I find it 'annoying'. The versions of Acronis TrueImage distributed by Seagate/Western Digital are incompatible (retail and other version are fine). I've obviously tried various Open Source options in the past but none come near the performance/compression balance that currently exists in this product and several other proprietary options.

While searching for a new external hard drive I came across the following...

http://www.phonenews.com/toshiba-acknowledges-macbook-woes-canvio-usb-3-0-hard-drives-18699/
http://www.theregister.co.uk/2012/07/23/100w_usb_power_delivery_specification/

I was recently playing around with the idea (and the practice) of turning Android phones into servers. It was not without its difficulties though. On most mobile computing devices there are forms of so called speed stepping technologies to help increase battery life. It's obviously re-appeared on Android as well but not without some of the problems that plagued other platforms. I once recall using a power control utility application on my old Android phone which underclocked it to such an extent (1/3 of it's full capacity) that I couldn't answer phone calls because there weren't enough CPU cycles for user space applications (such as unlocking the phone itself so that I could take a call) while playing music. Admittedly, the hardware specifications were minimal (it was an extremely early and inexpensive Android phone) but I would have thought a bit more thought would have gone into it prior to deployment (I 'accepted' the 'default settings' believing that that would have been adequate).

I obviously tried SSHDroid but like others I came across what were seemingly stability problems. Upon further examination though I found a lot about wireless and power control technologies. I think a lot of the problems that I was experiencing were as a consequence of power saving software (the above will likely make more sense to you now) which may have led to SSHDroid shutting down randomly and regularly. However, in my research I also came across a file transfer application called SwiFTP. In it, was option to shutdown most power saving behaviour. By enabling this, I've seen been able to achieve relatively stable connections (hours at a time).


You can lock up (somewhat) your Android phone by using firewall such as DroidWall.


Strongly suggest you have reliable equipment if you do ANY wireless experimentation. Obviously firmware changes can have significant impacts upon devices. One thing I underestimated though was how much of an impact 'raw power' played on connection stability. In the past (and recently) I was playing around with the idea of deliberately changing architecture to maintain wireless security (as opposed to relying on pure 'Faraday Cages' and other anti-TEMPEST techniques). By using the concepts of absorption, superposition, and reflection we can manipulate wireless signal pathways (very similar to some of the techniques that are used to develop low visibility/stealth aircraft). The results of these experiments indicate that putting this theory into practice will be less trivial than I had originally anticipated though (the intensity of a signal is conversely proportional to the inverse square of the distance between the source and destination if I recall my physics correctly).

at
Labels: , , , , ,

Tuesday, July 3, 2012

iPod Nano Device Sanitisation

Continuing my work on my, "Cloud and Internet Security" report. I'll revise my earlier estimate and say that its likely to be 500+ pages though the scope will depend on other factors. One of the issues that I wanted to explore was mobile device backup and sanitisation. In particular my iPod Nano. It sounds easy but if you haven't got a recent backup,  you're aware of how strongly iTunes and iPod devices are integrated, and have taken a look at the file directory structure of an iPod you'll realise just how awkward this can be.

Actual music files are stored on the device in the following format:
/media/sdd1/iPod_Control/Music/F[:number:]{2}/[:alpha:]{4}.mp3

The command, "file ZRJR.mp3" yields the following:
ZRJR.mp3: Audio file with ID3 version 2.2.0, contains: MPEG ADTS, layer III, v1, 128 kbps, 44.1 kHz, JntStereo


knoppix@system:~/iPod_Control/iTunes/iTunes Library.itlp$ file *
Dynamic.itdb:       SQLite 3.x database
Extras.itdb:        SQLite 3.x database
Genius.itdb:        SQLite 3.x database, user version 2
Library.itdb:       SQLite 3.x database, user version 23
Locations.itdb:     SQLite 3.x database
Locations.itdb.cbk: data

http://www.macworld.com/article/1139974/itunes_files.html

Plugging the device in under Linux and mounting as as a USB mass storage device works but so does iPod/iPhone Explorer under Windows as well (though they have some quirks that need to be fixed (I picked up some unhandled exception during my experimentation)). They remind me a lot of Explore2FS which we had to use in the early days of Windows/Linux interoprability. They also allow you to drag and drop though conversion from stored file format to MP3 will take some time. 

TeraCopy (a more optimal version of Windows Explorer's copy functionality) sufferred from a number of stalls and the only notification that I had was a balloon popup on the toolbar indicating that it was due to the process of conversion. Would have liked to see something more informative regarding progress.

Some of the main database table structures extracted from various onboard files are provided below (This was extracted by using a SQLite Browser/Manager and then exporting it to SQL code.).

CREATE TABLE album (pid INTEGER NOT NULL, kind INTEGER, artwork_status INTEGER, artwork_item_pid INTEGER, artist_pid INTEGER, user_rating INTEGER, name TEXT, name_order INTEGER, all_compilations INTEGER, feed_url TEXT, season_number INTEGER, is_unknown INTEGER DEFAULT 0, has_songs INTEGER DEFAULT 0, has_music_videos INTEGER DEFAULT 0, sort_order INTEGER DEFAULT 0, artist_order INTEGER DEFAULT 0, has_any_compilations INTEGER DEFAULT 0, sort_name TEXT, artist_count_calc INTEGER DEFAULT 0 NOT NULL, PRIMARY KEY (pid));

CREATE TABLE artist (pid INTEGER NOT NULL, kind INTEGER, artwork_status INTEGER, artwork_album_pid INTEGER, name TEXT, name_order INTEGER, sort_name TEXT, is_unknown INTEGER DEFAULT 0, has_songs INTEGER DEFAULT 0, has_music_videos INTEGER DEFAULT 0, PRIMARY KEY (pid));

CREATE TABLE avformat_info (item_pid INTEGER NOT NULL, sub_id INTEGER NOT NULL DEFAULT 0, audio_format INTEGER, bit_rate INTEGER DEFAULT 0, sample_rate REAL DEFAULT 0, duration INTEGER, gapless_heuristic_info INTEGER, gapless_encoding_delay INTEGER, gapless_encoding_drain INTEGER, gapless_last_frame_resynch INTEGER, analysis_inhibit_flags INTEGER, audio_fingerprint INTEGER, volume_normalization_energy INTEGER, PRIMARY KEY (item_pid,sub_id));

CREATE TABLE category_map (id INTEGER NOT NULL, category TEXT NOT NULL, PRIMARY KEY (id), UNIQUE (category));
CREATE TABLE composer (pid INTEGER NOT NULL, name TEXT, name_order INTEGER, sort_name TEXT, is_unknown INTEGER DEFAULT 0, has_music INTEGER DEFAULT 0, PRIMARY KEY (pid));
CREATE TABLE genre_map (id INTEGER NOT NULL, genre TEXT NOT NULL, genre_order INTEGER DEFAULT 0, is_unknown INTEGER DEFAULT 0, has_music INTEGER DEFAULT 0, artist_count_calc INTEGER DEFAULT 0 NOT NULL, album_count_calc  INTEGER DEFAULT 0 NOT NULL, PRIMARY KEY (id), UNIQUE (genre));

CREATE TABLE item (pid INTEGER NOT NULL, revision_level INTEGER, media_kind INTEGER DEFAULT 0, is_song INTEGER DEFAULT 0, is_audio_book INTEGER DEFAULT 0, is_music_video INTEGER DEFAULT 0, is_movie INTEGER DEFAULT 0, is_tv_show INTEGER DEFAULT 0, is_ringtone INTEGER DEFAULT 0, is_voice_memo INTEGER DEFAULT 0, is_book INTEGER DEFAULT 0, is_rental INTEGER DEFAULT 0, is_itunes_u INTEGER DEFAULT 0, is_digital_booklet INTEGER DEFAULT 0, is_podcast INTEGER DEFAULT 0, date_modified INTEGER DEFAULT 0, date_backed_up INTEGER DEFAULT 0, year INTEGER DEFAULT 0, content_rating INTEGER DEFAULT 0, content_rating_level INTEGER DEFAULT 0, is_compilation INTEGER, is_user_disabled INTEGER DEFAULT 0, remember_bookmark INTEGER DEFAULT 0, exclude_from_shuffle INTEGER DEFAULT 0, part_of_gapless_album INTEGER DEFAULT 0, chosen_by_auto_fill INTEGER DEFAULT 0, artwork_status INTEGER, artwork_cache_id INTEGER DEFAULT 0, start_time_ms REAL DEFAULT 0, stop_time_ms REAL DEFAULT 0, total_time_ms REAL DEFAULT 0, total_burn_time_ms REAL, track_number INTEGER DEFAULT 0, track_count INTEGER DEFAULT 0, disc_number INTEGER DEFAULT 0, disc_count INTEGER DEFAULT 0, bpm INTEGER DEFAULT 0, relative_volume INTEGER, eq_preset TEXT, radio_stream_status TEXT, genius_id INTEGER DEFAULT 0, genre_id INTEGER DEFAULT 0, category_id INTEGER DEFAULT 0, album_pid INTEGER DEFAULT 0, artist_pid INTEGER DEFAULT 0, composer_pid INTEGER DEFAULT 0, title TEXT, artist TEXT, album TEXT, album_artist TEXT, composer TEXT, sort_title TEXT, sort_artist TEXT, sort_album TEXT, sort_album_artist TEXT, sort_composer TEXT, title_order INTEGER, artist_order INTEGER, album_order INTEGER, genre_order INTEGER, composer_order INTEGER, album_artist_order INTEGER, album_by_artist_order INTEGER, series_name_order INTEGER, comment TEXT, grouping TEXT, description TEXT, description_long TEXT, collection_description TEXT, copyright TEXT, track_artist_pid INTEGER DEFAULT 0, physical_order INTEGER, has_lyrics INTEGER DEFAULT 0, date_released INTEGER DEFAULT 0, PRIMARY KEY (pid));

CREATE TABLE item_to_container (item_pid INTEGER, container_pid INTEGER, physical_order INTEGER, shuffle_order INTEGER);

CREATE TABLE video_info (item_pid INTEGER NOT NULL, has_alternate_audio INTEGER, has_subtitles INTEGER, characteristics_valid INTEGER, has_closed_captions INTEGER, is_self_contained INTEGER, is_compressed INTEGER, is_anamorphic INTEGER, is_hd INTEGER, season_number INTEGER, audio_language INTEGER, audio_track_index INTEGER, audio_track_id INTEGER, subtitle_language INTEGER, subtitle_track_index INTEGER, subtitle_track_id INTEGER, series_name TEXT, sort_series_name TEXT, episode_id TEXT, episode_sort_id INTEGER, network_name TEXT, extended_content_rating TEXT, movie_info TEXT, PRIMARY KEY (item_pid));


My guess is that they used the:

/media/sdd1/iPod_Control/Music/F[:number:]{2}/[:alpha:]{4}.mp3

structure as a means of avoiding filename clashes. iTunes basically manages various files and databases to keep track of where everything is. While its clear that there are alternatives they are not always perfect (I manually deleted a file from a my iPod and one of them was unable to correctly update the database. There has been some difficulty in the Open Source world with reverse engineering the details obviously and one of the solutions I've seen was adding/removing files using another program and then plugging it into iTunes to have it update its internal databases but you have to wonder about the logic of this as you're just adding another moving part to the already complex mix/interplay.) and have to be reverse engineered which can be time consuming and difficult as Apple continue to increase their use of anti-reverse engineering technologies (The more you dig the more you realise how drastically some organisation's thinking can veer off of what you believe to the most sensible/elegant option. During my sanitisation experiment a number of files kept on showing as being recoverable. My guess is that it has to do with the way in which files/directories are extracted and manipulated rather than remnant data though. You also obviously need to factor in design compromises and commercial considerations as well.).

CopyTransManager
CopyTransDoctor


It seems clear that the device itself is built off of industry standard software/hardware. Brilliantly integrated and packaged though. Underlying filesystem format is either HPFS or FAT32 depending on the operating system from which the device is setup from.

Tried using various wiping programs. However, it soon became clear that there  other issues at play here. Almost like there's another layer in between the operating system and the underlying filesystem. Not sure whether this is limited to my system (I have a Windows system that is dedicated to pure experimentation so I can't be entirely sure about it being true to other's experiences. I seemed to have no issues with interfacing with the underlying filesystem on Linux but that's not a 'standard environment'.) though.

AxCrypt - Shred and Delete
Shredding of ''????" failed,
A device attacked to the system is not functioning.

Eraser - Erase
Error - The file or directory is not a reparse point. (Exception from HRESULT: 0x80071126)

A file or directory can contain a reparse point, which is a collection of user-defined data. The format of this data is understood by the application which stores the data, and a file system filter, which you install to interpret the data and process the file. When an application sets a reparse point, it stores this data, plus a reparse tag, which uniquely identifies the data it is storing. When the file system opens a file with a reparse point, it attempts to find the file system filter associated with the data format identified by the reparse tag. If a file system filter is found, the filter processes the file as directed by the reparse data. If a file system filter is not found, the file open operation fails.

iPod Explorer - Simple Deletion
Obvious that one deletion of file is possible but it doesn't deal with updating the meta-data/database. It still appears in the menuing system but attempting to play results in nothing, not even an error.

Obviously, after deletion I tried various methods of file recovery to see just how effective some of the mechanisms I employed were. As with most standard FAT based filesystems if there were no other intervention 100% file recovery is almost always possible. You need to use a wiping program to truly defend your iPod or other mobile devices against attack from computer forensics programs. 

root@system:/media/sdc1# photorec /log /debug /d photorec
PhotoRec 6.11, Data Recovery Utility, April 2009
Christophe GRENIER
http://www.cgsecurity.org

Disk /dev/sdd - 7889 MB / 7523 MiB (RO) - Apple iPod
     Partition                  Start        End    Size in sectors
 1 P FAT32                    0   1  3   127 204 49    1926015 [USER'S IPOD]
168 files saved in photorec directory.
Recovery aborted by the user.
mp3: 148 recovered
tx?: 10 recovered
sqlite: 4 recovered
txt: 3 recovered
mov: 2 recovered
gz: 1 recovered
[ Quit ]

root@system:/media/sdc1# vim /etc/scalpel/scalpel.conf
# MPEG Video
mpg     y       50000000        \x00\x00\x01\xba        \x00\x00\x01\xb9
mpg     y       50000000        \x00\x00\x01\xb3        \x00\x00\x01\xb7

root@system:/media/sdc1# scalpel -v /dev/sdd1 -o /media/sdc1/scalpel
/dev/sdd1:   9.8% |********|  740.0 MB  2:05:55 ETAOPENING /media/sdc1/scalpel/mpg-1-0/00003091.mpg
/dev/sdd1:  10.0% |*******|  750.0 MB  2:06:05 ETAOPENING /media/sdc1/scalpel/mpg-1-0/00003093.mpg
CLOSING /media/sdc1/scalpel/mpg-1-0/00003091.mpg
^C^C^C
Kill signal detected. Cleaning up...
CLOSING /media/sdc1/scalpel/mpg-1-0/00003090.mpg
^C
Kill signal detected. Cleaning up...
CLOSING /media/sdc1/scalpel/mpg-1-0/00003089.mpg
^C
Kill signal detected. Cleaning up...
CLOSING /media/sdc1/scalpel/mpg-0-0/00000055.mpg
OPENING /media/sdc1/scalpel/mpg-1-0/00003092.mpg
CLOSING /media/sdc1/scalpel/mpg-1-0/00003092.mpg
/dev/sdd1:  10.1% |*********|  760.0 MB  2:04:56 ETACleaning up...
 Caught signal: Interrupt. Program is terminating early 

Erase Free Space (Eraser) option has reduced the significantly the chances of file recovery even though file integrity recovery indicators state (CopyTransDoctor) that 100% recovery is possible this is certainly not the case. My results indicate that while partial wiping is possible it requires specialised tools and/or knowledge (I've seen some mobile applications that are specifically designed for this purpose) and clearly if you have raw/jailbroken access to the file system you have many options/utilities available to you.

Not surprisingly given these issues, Apple and others have come up with their own wiping/encryption functionality. 


Simulated the following wiping (free space only. To run on the entire device just drop the digit.) algorithms,
- British HMG IS5 (3 passes)
- US DoD 5220.22-M (8-306./E)(3 passes)
by running the following:

dd if=/dev/zero of=/media/sde1/ipod.wipe
cat /dev/zero | tr '\0' '\1' | dd of=/media/sde1
dd if=/dev/urandom of=/media/sde1/ipod.wipe

Note that you will run into the 4GB filesystem limitation with FAT32 so just create a another file to get around this problem. Namely,

dd if=/dev/zero of=/media/sde1/ipod.wipe
dd if=/dev/zero of=/media/sde1/ipod.wipe1
dd if=/dev/zero of=/media/sde1/ipod.wipe2
etc...

It seemed to fairly obvious that a single wipe (British HMG IS5 (1 pass) or US DoD 5220.22-M (8-306./E)(1 pass) which basically amounts to a single pass of zeros) was enough to throw off the various programs that I was using to examine the device though (I was only using free and Open Source tools to simulate a worst case scenario.).

- scalpel
- photorec
- testdisk
- Recuva
- Glary Utilities
- diskinv

Not surprisingly, ports, and installations of Linux have been attempted. The main limitations seem to be the anti-reverse engineering technologies implemented by Apple though.



Given the fact that it basically is a USB storage device you can use a lot of standard utilities on the device such as DBAN and BCWipe.


Luckily recovery mode is there for you to be able to restore functionality most of the time (It's fairly easy to trigger. I obviously corrupted the filesystem a number of times through my experiments with dd and also attempting to reformat, use other programs, and operating systems on my device. Just plug it into a computer running iTunes and it'll do a complete reset of the device for you.).

http://www.tuaw.com/2010/12/31/discovering-ipad-nano-recovery-mode/

- as usual thanks to all of the individuals and groups who purchase and use my goods and services
http://sites.google.com/site/dtbnguyen/
http://dtbnguyen.blogspot.com.au/
at
Labels: , , , , ,
Subscribe to: Posts (Atom)

Dodgy Job Contract Clauses, Random Stuff, and More

- in this post we'll be going through dodgy job contract clauses. Ironically, many of which are actually unlawful and unenforceable on c...