I tried out a modern approach to image resampling: an end-to-end learned downsampler-then-upsampler. The application would be to downsample the image, compress it using a traditional image/video compression method, and then upsample the image back to its original size. This might be useful as part of a video pipeline.
This project ( code = https://github.com/sunwj/CAR paper = Learned Image Downscaling for Upscaling using Content Adaptive Resampler ) was found by looking at paperswithcode.com, where it presently ranks as the best method by far.
Here are some other example results you can zoom into:
When there are single-pixel features, it does struggle as you'd expect. But it does a good job preserving features that are ~2 pixels or larger.
To get this going on a Windows PC with an Nvidia graphics card, I installed:
- Rapid Environment Editor - Used to easily edit the PATH variable.
- Git Bash
- Latest Python 3
Make sure CUDA is in the Windows Path. For me I added:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin and
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\libnvvp.
Make sure Python3 is in the Windows Path. For me I added:
To install Ninja, I dropped the latest ninja.exe release file into the Python39 folder.
Then open up Git Bash and clone the repo and set up dependencies:
git clone firstname.lastname@example.org:sunwj/CAR.git cd CAR/adaptive_gridsampler pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio===0.8.1 -f pip install tqdm pip install scipy pip install numpy pip install matplotlib pip install scikit-image python setup.py build_ext --inplace cd ..
I then downloaded the pre-trained models from the link at https://github.com/sunwj/CAR and placed the files under a subfolder.
Now it's ready to test! The scripts by default support only .PNG losslessly compressed images.
python run.py --scale 4 --img_dir catid_images --model_dir models --output_dir output