Aim and scope

This sub-project aims to develop algorithms and implementation for efficient media analysis and processing. Arkiv Digital has 50 million high-resolution images and the total amount of image data is approximately 0,5 petabytes. The amount of data poses significant challenges on data storage, image processing, and data transfer. Further, Sony Mobile would like to do advanced image processing on mobile devices with limited execution capabilities and battery power. The challenges are mainly in these areas:

• Processing of the image data, e.g., image analysis, image classification, blur detection, object identification, and pattern recognition, classification, and matching. Here are the real-time image analysis and classification performance important, as outlined in IC1.
• Compression of images, e.g., based on quality metrics, in order to reduce data sizes when sending images to clients while also keeping high quality. Also here are real-time aspects of significant importance since the response time an image request to image delivery strongly influences the customer satisfaction.
• Efficient implementation of image algorithms on heterogeneous systems, e.g., dividing work between CPUs and GPUs. Further, mobile platforms pose additional challenges for efficient parallel algorithms, e.g., partitioning, mapping, optimizing, etc., and new frameworks may need to be developed.
• Develop improved and specialized zooming and text enhancement techniques

Proposed work

The work will mainly be centered around a model of digital camera pipeline, where image data is processed in various steps. For example, we work on how to convert images from the image sensor data to RGB, and the to other types of high-quality compressed image formats. Further, we work on techniques for quality assessment as well as to enhance readability and understandability of historical handwritten
documents, and on various image classification and recognition techniques. Finally, we will have focus on how to implement these techniques for efficient execution on heterogeneous and mobile platforms.

Evaluation/methodology

The evaluation is done mainly through theoretical/algorithmic analysis, prototype development, experimental evaluation, and measurements. Typical metrics that we will use are, e.g., classification performance, data sizes (compression ratios), execution time,
and memory requirements.