Kiyasta Haske na Waje Mai Zurfi: Hanyar Tushen CNN daga Hotuna Guda na LDR

Table of Contents

1. Gabatarwa

Dawar da hasken wuri daga hoto guda matsala ce ta asali amma ba ta da tabbas a cikin hangen nesa na kwamfuta, mai mahimmanci ga aikace-aikace kamar gaskiyar daɗaɗɗe (AR), zane-zane na tushen hoto, da fahimtar wuri. Takarda "Kiyasta Haske na Waje Mai Zurfi" ta magance wannan ƙalubale musamman ga wuraren waje ta hanyar gabatar da hanyar tushen Cibiyar Jijiyoyin Convolutional (CNN) don hasashen Haske mai ƙarfi (HDR) na waje daga hoton Ƙaramin Ƙarfi (LDR) guda. Sabon abu na asali ya ta'allaka ne a kan keta buƙatar ɗaukar taswirar muhalli ta HDR kai tsaye ta hanyar amfani da babban dataset na panoramas na LDR da kuma tsarin sama na tushen zahiri don samar da dataset na horo na roba na nau'i-nau'i na hoto-haske.

2. Hanyar Aiki

Tsarin da aka gabatar ya ƙunshi manyan matakai biyu: shirya dataset da horarwa/bayanan CNN.

2.1. Ƙirƙirar Dataset & Daidaita Tsarin Sama

Marubutan sun kewaye rashin manyan datasets na LDR-HDR masu haɗin gwiwa ta hanyar amfani da tarin panoramas na waje. Maimakon amfani da panoramas kai tsaye a matsayin maƙasudai na HDR, sun daidaita sigogin tsarin sama na Hošek-Wilkie zuwa yankunan sama da ake gani a cikin kowane panorama. Wannan tsarin, wanda wakili ne na ƙaƙƙarfan sigogi $\Theta = \{\theta_{sun}, \theta_{atm}, ...\}$, yana bayyana matsayin rana, yanayin yanayi, da turbidity. Wannan matakin yana matsawa cikakken bayanin haske mai siffar zobe zuwa wani vector mai ƙarancin girma, mai ma'ana ta zahiri wanda CNN zai iya koya. An ciro hotuna da aka yanke, masu iyakacin filin kallo daga panoramas don zama shigarwar CNN, suna ƙirƙirar nau'i-nau'i na horo $(I_{LDR}, \Theta)$.

2.2. Tsarin CNN & Horarwa

An horar da CNN don yin koma baya daga shigarwar hoton LDR zuwa vector na sigogin tsarin Hošek-Wilkie $\Theta$. Cibiyar sadarwa tana koyon haɗin gwiwa mai rikitarwa tsakanin alamun gani a cikin hoton (launin sama, alamun matsayin rana, inuwa, sautin wuri gabaɗaya) da yanayin haske na zahiri na asali. A lokacin gwaji, idan aka ba da sabon hoton LDR, cibiyar sadarwa tana hasashen $\hat{\Theta}$. Ana iya amfani da waɗannan sigogi tare da tsarin Hošek-Wilkie don haɗa cikakken taswirar muhalli ta HDR, wanda daga baya ake amfani da shi don ayyuka kamar shigar da abubuwa na roba masu kama da gaskiya.

3. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Tsarin sama na Hošek-Wilkie shine tsakiyar hanyar. Tsarin sama ne na bakan wanda ke lissafta radiance $L(\gamma, \alpha)$ don wani ma'ana na sama da aka ayyana ta kusurwar zenith $\gamma$ da kusurwar zenith ta rana $\alpha$. Tsarin ya haɗa da wasu ƙididdiga na ƙwararru don tarwatsa yanayi. Tsarin daidaitawa ya haɗa da rage kuskuren tsakanin fitarwar tsarin da pixels na sama na panorama da aka lura don warware mafi kyawun saitin sigogi $\Theta^*$:

$$\Theta^* = \arg\min_{\Theta} \sum_{p \in SkyPixels} || L_{model}(p; \Theta) - I_{panorama}(p) ||^2$$

Wannan $\Theta^*$ da aka dawo da shi yana aiki ne a matsayin gaskiyar ƙasa don horar da CNN. Aikin asara don horar da CNN yawanci asara ne na koma baya kamar Kuskuren Matsakaicin Matsakaici (MSE) ko bambance-bambancen ƙarfi kamar asarar Smooth L1 tsakanin sigogin da aka hasashen $\hat{\Theta}$ da gaskiyar ƙasa $\Theta^*$.

4. Sakamakon Gwaji & Ƙima

4.1. Ƙimar Ƙididdiga

Takardar tana kimanta hanyar akan dataset na panorama da kuma wani saiti na taswirorin muhalli na HDR da aka ɗauka. Ma'auni mai yiwuwa ya haɗa da kuskuren kusurwa a cikin matsayin rana da aka hasashen, kuskure a cikin sigogin haske, da ma'auni na tushen hoto don abubuwan da aka zana. Marubutan sun yi iƙirarin cewa hanyarsu "ta fi naɗaɗɗiyar mafita ta baya," wanda zai haɗa da hanyoyin da suka dogara da alamun hannu kamar inuwa [26] ko rarrabuwar hoto na ciki [3, 29].

4.2. Sakamako na Halitta & Shigar da Abu na Roba

Mafi kyawun nunin shi ne shigar da abubuwa na roba masu kama da gaskiya cikin hotunan gwaji. Hoto na 1 a cikin PDF a zahiri yana nuna wannan tsarin: ana ciyar da shigarwar hoton LDR zuwa CNN, wanda ke fitar da sigogin sama da aka yi amfani da su don sake gina taswirar muhalli ta HDR. Daga nan sai a zana wani abu na roba a ƙarƙashin wannan hasken da aka kiyasta kuma a haɗa shi cikin hoton asali. Sakamako masu nasara suna nuna daidaitaccen shugabanci na haske, launi, da ƙarfi tsakanin abu na roba da wurin gaskiya, suna tabbatar da daidaiton hasken da aka kiyasta.

5. Tsarin Bincike: Fahimta ta Asali & Kwararar Ma'ana

Fahimta ta Asali: Hazakar takardar ita ce hanyarta ta daidaita bayanai. Maimakon tunkarar aikin da ba zai yiwu ba na tattara manyan nau'i-nau'i na LDR-HDR na zahiri, marubutan sun yi amfani da wayo da panoramas na LDR da suke akwai ta hanyar amfani da tsarin zahiri na sigogi a matsayin "gada" don samar da kulawar HDR mai ma'ana. Wannan yana tunawa da canjin tsarin da ayyuka kamar CycleGAN suka ba da damar, waɗanda suka koyi taswirori tsakanin yankuna ba tare da misalan haɗin gwiwa ba. A nan, tsarin Hošek-Wilkie yana aiki a matsayin malami mai ilimin kimiyyar lissafi, yana tace haske mai rikitarwa zuwa wakilcin da za a iya koyawa.

Kwararar Ma'ana: Ma'ana tana da inganci amma tana dogara ne akan zato mai mahimmanci: cewa tsarin Hošek-Wilkie yana da isasshen daidaito da gabaɗaya don wakiltar yanayin haske daban-daban a cikin panoramas na horo. Duk wani son zuciya na tsari a cikin tsarin ko tsarin daidaitawa an toshe shi kai tsaye cikin "gaskiyar ƙasa" na CNN, yana iyakance babban iyakar aikin sa. Kwararar ita ce: Panorama (LDR) -> Daidaita Tsarin -> Sigogi (Gaskiya ta Ƙaƙƙarfan) -> Horar da CNN -> Hoton Guda -> Hasashen Sigogi -> Haɗin HDR. Misali ne na gargajiya na "koyon juzu'in tsarin gaba."

Ƙarfi & Aibobi: Babban ƙarfi shi ne aiki da iya aiki. Hanyar tana da horo kuma tana samar da sakamako na zamani. Duk da haka, aibobinta suna cikin ƙirar sa. Na farko, an iyakance shi ga yanayin sama mai tsabta, hasken rana wanda Hošek-Wilkie ya ƙirƙira. Sama mai girgije, yanayi mai ban mamaki, ko tasirin ramin birni tare da haske kaikaice mai rikitarwa ba a sarrafa su da kyau ba. Na biyu, yana buƙatar sama da ake gani a cikin hoton shigar - iyaka mai mahimmanci ga yawancin hotunan da masu amfani suka ƙirƙira. Hanyar, kamar yadda aka bayyana, mai daidaita tsarin sama ne, ba mai kiyasta hasken wuri ba.

Fahimta Mai Aiki: Ga masu aiki, wannan aikin babban darasi ne na yin amfani da kulawar kaikaice. Abin da za a ɗauka shi ne koyaushe a nemi kadarorin bayanai da suke akwai (kamar ma'ajin panorama) da ilimin yanki (kamar tsarin zahiri) waɗanda za a iya haɗa su don ƙirƙirar siginonin horo. Ci gaban wannan ra'ayi na gaba, kamar yadda aka gani a cikin ayyukan baya daga Binciken Google da MIT, shine matsawa sama da tsarin sama na sigogi zuwa hasashen taswirar muhalli ta HDR mara sigogi ta ƙarshe ta amfani da ƙirar ƙira mafi ƙarfi (kamar GANs ko NeRFs) da ma mafi girma, datasets masu banbanci, mai yuwuwa haɗa bayanan lokaci daga bidiyoyi.

6. Hangar Aikace-aikace & Hanyoyin Gaba

Aikace-aikacen nan take shine a cikin gaskiyar daɗaɗɗe don shigar da abu na waje mai gaskiya a cikin daukar hoto da fim (misali, don tasirin gani). Hanyoyin gaba sun haɗa da:

• Faɗaɗa Tsarin Haske: Haɗa tsarin don sama mai girgije, magriba, da hasken wuta na wucin gadi don sarrafa yanayi mai faɗi.
• Kiyasta Ba tare da Sama ba: Haɓaka fasahohin da za su iya fahimtar haske daga filayen ƙasa, inuwa, da inuwar abu lokacin da aka rufe sama, watakila ta hanyar haɗa da ƙididdigar geometry a sarari.
• Haske Mai Ƙarfi: Tsawaita hanyar zuwa bidiyo don kiyasta haske mai canzawa lokaci, mai mahimmanci ga daidaitaccen AR a cikin wurare masu ƙarfi.
• Haɗin kai tare da Zane na Jijiya: Haɗa kiyasta haske tare da filayen radiance na jijiya (NeRF) don sake gina wuri tare da sake haskakawa, hanya da cibiyoyi kamar UC Berkeley da NVIDIA ke bi.
• Inganta Na'urar: Tsarin cibiyar sadarwa mai sauƙi don kiyasta lokacin gaskiya akan na'urorin hannu, yana ba da damar aikace-aikacen AR na masu amfani.

7. Nassoshi

1. Hold-Geoffroy, Y., Sunkavalli, K., Hadap, S., Gambaretto, E., & Lalonde, J. F. (2018). Deep Outdoor Illumination Estimation. arXiv preprint arXiv:1611.06403.
2. Hošek, L., & Wilkie, A. (2012). An analytic model for full spectral sky-dome radiance. ACM Transactions on Graphics (TOG), 31(4), 1-9.
3. Barron, J. T., & Malik, J. (2015). Shape, illumination, and reflectance from shading. IEEE transactions on pattern analysis and machine intelligence, 37(8), 1670-1687.
4. Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2223-2232).
5. Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020). NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In European conference on computer vision (pp. 405-421). Springer, Cham.
6. Google AI Blog: "Looking to Lift: A New Model for Estimating Outdoor Illumination" (Wakilin binciken masana'antu na gaba).