Real-Time Tone Mapping for High-Resolution HDR Images

International Conference on Cyberworlds 2008Real-TimeToneMappingforHigh-ResolutionHDRImagesHanliZhaoStateKeyLabofCAD&CGZhejiangUniversity,anlizhao@ogangJinStateKeyLabofCAD&CGZhejiangUniversity,in@bingShenDepartmentofComputerandInformationScienceIndianaUniversity-PurdueUniversityIndianapolis,USAshenjian@tractHighdynamicrangerenderingattemptstotakeanHDRimageandproghseveraltonemap-pingoperatorshavebeenproposedinrecentyears,noeval-uationhasyetbeenundertakpaper,webeginwithourveralexperimentalresultsusingeightGPU-basedtonemappingoperatorsarepresentedtoevaluatewhichoneisbetterwithregardtorun-ningeffiimplementationofthePattanaikoperation,webelievethatmanyreal-timeapplications,includingHDRvideoplayeranden-vironmentmappingwithHDRtexturesingames,willbenefitfromournovelapproach.1IntroductionIntherealworld,itiswell-knownthatghdynamicrange(HDR)stremendousprogressinimagingtechnology[8,28,37,26]rast,ghrecentresearchesindisplaytechnologyhavesuggestedthatHDRdisplaysareonthehorizon[16],ntyears,agreatnumberoftonemappingopera-tors(TMOs)havebeendevelopedtodisplayHDRdataonlowdynamicrange(LDR)scanbeperformedasapost-processingstagetoanygameen-gine,real-timerenderingsystem,r,mostoftheseTMOsarecompuately,programmablegraphicshardwareisgettingmoreandmorepowerfulforgeneralpurposecomputing(GPGPU)[15,3],cently,someTMOshavebeenimplementedonGPUwithgoodreproductionquality[14,33,35],whiletheseapproachesarppingalgorithmscanbeclassifiedintotwobroadcategories:aloper-atorsaspecificmappingtacticisusedforeachpixel,tance,PattanaikandYee[29]extendedtheTMOin[30]operatorsreducecontrastsbasedongloballyderivedquantities,whital.[11]proposedafast,highqualityglobaloperatorbasedonlogarithmiccompressionoflumi-nancevalues,ngadvantageoftheprogrammabilityofmoderngraph-icshardware,icularly,theGPU-basedPatpermakesthefollowingthreecontributions:•NovelGPUimplementationsoftwostate-of-the-artTMOswithreal-timeperformanceispresented.•ExperimentalevaluationisundertakentoexplorewhichTMOisfasterforhardwareimplementation.•Manyreal-timeapplications,includingHDRvideo978-0-7695-3381-0/08 $25.00 © 2008 Crown CopyrightDOI 10.1109/CW.2008.12256

playerandenvironmentmappingwithHDRtextures,willbenefin3de-scribesourapproachindetail,y,Section5concludesthepaper.2RelatedWork2.1Real-TimeTMOsTumblinandRushmeier[36]firstproposedtheideaoftonemappingbasedonhumanperceptionarreadersto[9]sectionwewilllresearchershaveproposedaccelerationmeth-odsinordertoimprovethecomputationalperformanceofTMOs[12,34,6,1].Somemethodscurrentlydoachieveinteractiverates,whilequalightetal.[14]or[32]andproposedafastalgorithmforthephotographiczyketal.[18]combinedthereproductionofpercep-tualeffectswithinreal-timetonemapping,whereasColbertetal.[7]al.[33]presentedahardwareimplementationoftheAshikhminoperator[2]andproposedaneffirmannandHensley[35]acceleratedWard’shistogramadjustmenttonemappingoperator[22]eperformance,however,isstillnotsatisfistonemappingeffectsofHDRimageCathedral(fromlefttorightandfromtoptobottom):inputimage,simplelogarithmicoperator,Reinhardlocalopera-tor,Reinhardglobaloperator,Ashikhminlo-caloperator,Ashikhminglobaloperator,Du-randoperator,Dragooperator,tal.[10]ranpreferenceandnaturalnessevaluationexperi-mentstomeasurethedisaetal.[38]askedobsertal.[23]presentedre-sultsofapsychophysicalinvestigationtovalidatethatanHDRdisplayiscatal.[24]validatedsixTMOsagainstlinearlymappedHDRscenesonaHDRdisplaybypresentingresultsofaseriesofpsychophysicalˇıketal.[4]´herexperimentshavebeenpresentedtoevaluateHDRimagerenderingalgorithms[17,20,19,21].Theseexperiments,however,ardetal.[31]showedacom-prehepaper,wepresentseveralex-perimentalresultstofindoutwhichisbetterforhardwareimplementationwithregardtoframerates,ult,manyreal-timeapplicationsrelatedtoHDRimagingwillbenefitfromthispaper.257

3OurApproachInthissection,weintroduceournovelGPUimraccelerationtechnique,bothstate-of-the-artoperatorsareabletoachievereal-timeperformance;3.1Real-TimePattanaikOperatorSimplescalingtofitthehighdynamicrangeimagedatatotherangeoftheavailabledisplaysresultsinlossofimagedetail(localcontrast)ailindarkerareabecomeindistinguishablefrlocaladaptationalgorithmthePattanaikOp-eratorcompresseshighintensitychangeswhilepreservingintensitychangesduetothechangeinsurfacerefldetectsthepresenceofhighcontrastedgesandremovetheinfluenceoal.[5]proposedareal-timedetail-preservinglo-caltonemappingalgorithm[13]ethodtakesmuchtimetoconstructthebilateralgridusingaGPUscatteroperation[15].Incontrast,heker-nelsizewith7pixelsisquiteenough,udrreadersto[30]sentedcodeishighlypar-allel,enablingeffi-codeoftheDragooperator.3.2Real-TimeDragoOperatorUnlikethePattanaikoperator,theDragooperatorper-formsaglobaltonemappingtechniquebasedonlogarith-miccompressionofluminancevalues,owerfunctionisintroducedtoadaptivelyvarylogarithmicbases,-ngadvantageoftheparallelismofgraphicshardware,wefurtherimproveitseffi-basedmethodgeneratesequaortimeconsumingcomponentisthecalcula-tionsofglobalstatisticquantities(maximumandlogaver-age).Goodnightetal.[14]calcr-formedrepeateddownsamplings,r,theirmethodrequiresapproximatelylog2npasses,tetal.[7]calculatedthesevaluesonCPUtoimprovethecom-putationeffir,graphicshard-warenativelysupportsthealphablendingoffloating-pointdata[3].Thestatisticquantitiescanbecalculatedbyper-formingascatteroperationusingvertexshader[25]averageoperation,wefirstcleartherendertargetasazero,thenaccumulatetherendertargetcolorbysettingtheblendoperationasADD,theaveragevalueisfinallycalculat-larly,forthemaximum(minimum)operation,wefirstcleartherendertargetasanegative(positive)infinity,thenthe258

maximum(minimum)valueiscomputedinasinglepassbysettingtheblendoperationasMAX(MIN).AsGPUsup-portsdifferentblendingoperationsforcolorchannelsandalphachannel,proach,however,imple-mentedbothalgorithmsandfievethattheGPUmethodtocalculatetheglobalquantitieswilloutper-fofall,luminanc-terloggingtheluminanceonGPU,eGPU-basedDragooperatorperformsthetonereproductiont,thegammacorrectionisappliedtothecompressedcolor.4ExperimentalEvaluationWehavebenchmarkedeightGPU-basedTMOsonaPCwitha2.40GHzIntelCore2Duo6320CPU,2GBmainmemory,anNVIDIAGeforce8800GTSGPU,384MBgraphicsmemory,nowledge,notesthasbeenundertakenforevaluatingtheefficiencyofsomanyGPU-basedTMOs.(a)Inputimage(b)Dragooperator(c)Pattanaikoperator4.1EightGPU-basedTMOsInadditiontothePattanaikOperatorandtheDragoOperatorwithGPUimplementationswhichhavebeenin-troduced,ioustonemappingeffectsusingtheseGPU-basedTMOsareshowninfithmicOperatorisincludedinourexperimentsbecauseitisoneofthemoststraightforwardtechniquesthatpthelogarithmardLocalOperator[32,7]firstappliesascal-ingthatanalogoustosettingexposureinacameraandthenaccomplishesdyrdGlobalOperator[32]insteadusesLwhitetominLocalOperator[2,33]firstestimateslocaladaptionlevel,aeratorfollowsfunctionalityofhumppingresultsofHDRim-agesMirror,Memorial,tcolumnshowstheinputimages,themid-columnshowstheeffectsusingtheDragooperator,whereasthminGlobalOperator[2]Operator[13]reducestheoverallcontrastwhilepreservinglocaldetoduc-ingthebilateralgrid,Chenetal.[5]approximatedthebilat-eralfilteronGPU,rpaper,theyalsopre-sentedreal-tiytheGPU-basedbilateralfiltertothebaselayer,andreducethecontrastau-tomaticallywithregardtoimagecontrast(maximumandminimum).259

Frames Per SecondResolutionReinhardReinhardAshikhminAshikhminLogarithmicDurandDragoPattanaikLocalGlobalLocalGlobalMirror346×59543Memorial512×76811878151Desk644×874836176Cathedral767×144130128Office2000×13122Price3272×fiatonlyourimplementationofthePattanaikoperatorcanachievereal-timeperformanceforhigh-resolutionHDRimages.4.2ExperimentalResultsandDiscussions5.2EnvironmentTexturesMappingwithHDRTheeffitestedTMOscanachievereal-timeframeratesontheimageCathedral,whichhasaresolutionof767×eincreaseoftheimagereso-lution,theReinhardlocaloperator,theAshikhminlocalop-erator,theDurandoperator,andtheDragooperatorfailtooffermorethan10FPSontheimageOffice,whichhasaresolutionof2000×imagePriceWest-ernwithmorethan4mega-pixels,2andfiboxhasbeenusedin3Dgameengines[27],andherewedemonsisonofenvironmentmappingwithoutandwithtonemappingtech-niqueispresentedinfitapplyingtonemap-pingtechnique,imagepixelsindarkareaandwhitearealosetheirdetails,whereastherightimagecanbedisplayedonlowdynamicrangemonitors.5ApplicationsAvarietyofreal-timeapplicationswillbenefisection,wedemonstratetheHisonofenvironmentmap-pingwithHDRtextureswithoutandwithtonemappingtechnique.5.1Real-TimeHDRVideoPlayer6ConclusyerrendersthecompressedHDRvideostream[26]toafloating-pointtextureandthenperformsreaepaper,wepresentnovelGPUiatourproposedalgomentalevaluationisundertakentoexplorewhichTMOisfasterfor260

(a)Dragooperator(b)ppingresultsoftheimagesOffierimentalresultsdemonstrateboththefeasibilityandeffionemappingtechniquescanbeperformedasapost-processingstagetoanygameengine,real-timerender-ingsystem,ordigitalvideoplayer,webelievethatmostofthoseapplicationswillbenefitdesignofthealphablendstagecanonlysupportlimitedandfixedfunctionblendingoperationsandthusthecalculationdliketoseeamorepowerfulandprogrammableblendstageextendingcurrentfunctionality,nces[1],r,,’03:Proceedingsofthe14thEurographicswork-shoponRendering,pages38–aphicsAssociation,2003.[2]’02:Proceedingsofthe13thEurograph-icsworkshoponRendering,pages145–aphicsAssociation,2002.[3].,25(3):724–734,2006.ˇık,,n,[4]´eedingsof14thPacificConferenceonComputerGraphicsandApplications,pages35–alTaiwanUniversityPress,2006.[5],,RAPH’07:ACMSIGGRAPH2007papers,pages171–,2007.[6],,s,eedingsofthe12thEurographicsWorkshoponRenderingTechniques,pages313–er-Verlag,ledgementsTheauthorswouldlirkwassupportedbytheNationalNaturalScienceFoundationofChina(GrantNos.60533080),theChina863Program(GrantNo.2006AA01Z314),theKeyTechnologyR&DProgram(GrantNo.2007BAH11B03),andtheProgramforNewCenturyExcellentTalentsinUniversity(-05-0519).261

[7]t,rd,epresent.,18(5):387–396,2007.[8]RAPH’97:ACMSIGGRAPH1997papers,pages369–,1997.[9],rs,,:eoftheArtReports,Eurographics2002,pages101–aphicsAssociation,September2002.[10],s,wski,andH.-RAPH’03:ACMSIGGRAPH2003Sketches&Appli-cations,pages1–,2003.[11],wski,,,22(3):419–426,2003.[12]-ceedingsoftheEurographicsWorkshoponRenderingTech-niques2000,pages219–er-Verlag,2000.[13]lateralfiRAPH’02:Proceedingsofthe29thannualconferenceonComputergraphicsandinteractivetechniques,pages257–,2002.[14]ght,,y,-tera’03:Proceedingsofthe14thEurographicsworkshoponRendering,pages26–-graphicsAssociation,2003.[15]RAPH’05:ACMSIGGRAPH2005Courses,,2005.[16],W.L.A.,-posiumDigestofTechnicalPapers,34(1):1450–1453,2003.[17]ndconcernsofCIETC8-08:eedingsoftheSocietyofPhoto-OpticalInstrumentationEngineers(SPIE),volume5668,pages148–156,2004.[18]yk,wski,andH.-’05:Proceed-ingsofthe21stspringconferenceonComputergraphics,pages195–,2005.[19],n,rImagingConference,pages8–13,2005.[20],chi,n,rImagingConference,pages315–320,2004.[21],chi,,n,t.,4(2):Article9,27pages,2007.[22],ier,ansactionsonVisualizationandComputerGraphics,3(4):291–306,1997.[23],rs,plays:ternationalConferenceonSystems,ManandCybernetics,3:2777–2782,2004.[24],rs,anko,.,24(3):640–648,2005.[25],,¨uger,l,araju,,y,:RAPH’04:ACMSIGGRAPH2004CourseNotes,,2004.[26]k,v,wski,andH.-.,25(3):713–723,2006.[27]alve’RAPH’06:,2006.[28]namicrangeimaging:,01:1472,2000.[29]’02:Proceedingsofthe18thspringconferenceonComputergraphics,pages83–,2002.[30]aik,n,,RAPH’00:Proceedingsofthe27than-nualconferenceonComputergraphicsandinteractivetech-niques,pages47–,2000.[31]ansactionsonVisualizationandComputerGraphics,11(1):13–24,2005.[32]rd,,y,.,21(3):267–276,2002.[33],,l,nthou,07:,2007.[34],nger,andH.-erGraphicsForum,19(3):301–312,2000.[35]fi3D’07:Proceedingsofthe2007symposiumonInteractive3Dgraphicsandgames,pages33–37,NewYork,NY,USA,.[36].,13(6):42–48,1993.[37]-hdr:abackwards-compatible,-GRAPH’05:ACMSIGGRAPH2005Courses,,2005.[38]a,,wski,andH.-nVisionandElectronicImagingX,IS&T/SPIE’s17thAnnualSymposiumonElectronicImag-ing,volume5666,pages192–,2005.262