OpenMP multithreading


Posting rules: It shouldn't need saying, but... play nice. Please keep your discussions civil. You can disagree, just don't be disagreeable. And, of course, all of the usual stuff like no spamming. Tex adds: I'll be rigorously enforcing this as we go along. We're probably going to be a small community in a little lifeboat, so we can't have members at each others' throats. This is for the sake of the project as a whole. So when you post, pretend you're speaking in person with your very wealthy auntie who has always treated you wonderfully and currently lists you prominently in her will. I won't be tossing anyone out of the forums because we are all in this together (except spammers: immediate membership cancelation), but I'll delete suspect posts right away.


9 posts / 0 new
Last post
ktgw0316
OpenMP multithreading

There's an OpenMP multithreaded version of dcraw in UFRaw source code. I made patch for the latest dcraw.c from the code and include it in LightZone v4.1.0~beta8. 

 

I compared the new version and normal one on my Ubuntu box. Machine spec was:

% grep -m1 '^model name' /proc/cpuinfo
model name : AMD Athlon(tm) II X4 645 Processor
% grep '^Mem' /proc/meminfo
MemTotal:       10240896 kB
MemFree:          156580 kB

 

and results of time measurements with Canon and Nikon raws:

 

% time ./dcraw canon_eos_5d_mark_iii_01.cr2

./dcraw canon_eos_5d_mark_iii_01.cr2  6.78s user 0.18s system 99% cpu 7.011 total

% time ./dcraw_mt canon_eos_5d_mark_iii_01.cr2

./dcraw_mt canon_eos_5d_mark_iii_01.cr2  6.99s user 0.18s system 162% cpu 4.409 total

 

% time ./dcraw nikon_d7000_04.nef

./dcraw nikon_d7000_04.nef  5.05s user 0.14s system 99% cpu 5.195 total

% time ./dcraw_mt nikon_d7000_04.nef

./dcraw_mt nikon_d7000_04.nef  5.27s user 0.21s system 156% cpu 3.505 total

 

Obviously, multithreaded version is faster.

 

Masahiro from dev-team

Francois_C
Yesterday, I compiled the LZ

Yesterday, I compiled the LZ DCRaw 9.19 version I got from this site with GCC 4.8.2 Windows 64 for my i7. I tried -march=corei7-avx and -march=core-avx-i. The first one seems to be better than the other, but unsure whether faster than the generic version!

Is the patch included in this version?

ktgw0316
Re:

The patces are not included in LightZombie/dcraw by Doug, but are included in this upstream repo:

https://github.com/Aries85/LightZone/tree/master/lightcrafts/coprocesses...

"dcraw_lz.c" is the patched version. You need to build with "-fopenmp" flag to enable the multi-threading.

 

Masahiro

Francois_C
Great!

Great!

I'll try as soon as possible!

Thank you.

Done.

(gcc -march=corei7-avx -o dcraw -O4 dcraw_lz.c -lm -g -DNODEPS -DLIGHTZONE -fopenmp -lws2_32)

Seems to be much faster!

Francois_C
But...

I noticed some errors with the binary I built:

when processing X-Trans files, i get images with a green transparent rectangle, generally towards top left. Not noticed in my first tries.

I saw there is a dcraw.dll too. Does it need to be rebuilt?

Photonoxx
I have the same issue with

I have the same issue with linux 64bit and beta 10 (repository version for ubuntu 14.04). I have opened a subjec under beta linux section about that because i try under win xp 32bits without issues (but I test it less than linux version)

 

ktgw0316
I've just updated the dcraw

I've just updated the dcraw_lz.c. Could you check if that works?

There were bugs that cause race conditions in multithreading patch for xtrans_interpolate.

Photonoxx
I launch via command line

I launch via command line about ten time, and I never get artifacts...

I've place this newly build dcraw_lz in /opt/lightzone and I've continued testing inside LZ too, and after opening 10 different pictures, I don't get any artifact.

 

 

So the matter seems to be solved. For the moment at least. 

 

Thanks a lot for the fix ! 

Francois_C
Discovering this version

Discovering this version today.

Seems to be great. Compiled on Linux with:

gcc -march=corei7-avx -o dcraw -O4 dcraw_lz.c -lm -g -DNODEPS -DLIGHTZONE -fopenmp

Test image:

https://www.flickr.com/photos/fcollard/14884876617/

[Edited] Also works on Windows 64. No green rectangles yet. Very fast.