Prokudin-Gorskii Image Alignment Project

This project involves the development of an algorithm that processes early color photographs, such as those from the Prokudin-Gorskii collection, by aligning and merging the red, green, and blue channels. The goal is to recreate the full-color image from separate grayscale channels. The process includes alignment using structural similarity (SSIM), contrast adjustments, and edge-based image alignment to achieve the best quality.

Approach

1. Channel Extraction: The three color channels (blue, green, and red) are extracted from a single grayscale image. These channels are then aligned with respect to the blue channel, which acts as the reference. The alignment is done based on the similarity of the images, measured using either SSIM or normalized cross-correlation (NCC).
2. Alignment Using SSIM and Edge Detection: SSIM is calculated between the reference and target channels to find the best alignment. Edge detection using the Sobel operator is employed to focus on structural details, improving the accuracy of the alignment.
3. Parallelization for Speed: The alignment process is parallelized to improve the computation time, especially when dealing with large offsets. Multiple shifts are tried in parallel using ThreadPoolExecutor.
4. Image Pyramid for Coarse-to-Fine Alignment: A multi-level pyramid approach is used to handle large images. The algorithm starts with downsampled versions of the image, finding the best alignment at each level and refining the alignment as it moves to higher resolutions.
5. Automatic Cropping and Contrast Adjustment: After alignment, automatic cropping removes the borders where misalignment occurs. Contrast is then adjusted so that the image has a balanced intensity range.

Problems Encountered and Solutions

Alignment Failures: In some images, significant misalignment occurred due to large offsets or significant rotations, which the algorithm does not currently handle. A possible improvement would be to implement rotational alignment and scaling.
Border Misalignment: Due to imperfect alignment, there were often visible borders. The solution was to implement automatic cropping based on image gradients, which removes the problematic regions.
Contrast Issues: Some images appeared washed out or too dark. The automatic contrast adjustment normalized the pixel intensities, improving the visual quality without requiring manual intervention.

Bells and Whistles Implemented

1. Automatic Cropping: Automatically detects and removes borders where the channels do not align properly, based on gradients (implemented in crop_borders).
2. Automatic Contrast Adjustment: Adjusts the image intensities to scale from 0 to 1 for improved visual contrast (implemented in automatic_contrast).
3. Edge-based Alignment (Sobel Detection): Uses Sobel edge detection to align images based on structural features rather than raw pixel values, improving the accuracy of alignment (implemented in align_images_ssim).