Ƙididdigar Haske na Cikin Gida Mai Zurfi: Sabuwar Hanyar Haske Mai Bambancin Wuri

1. Gabatarwa

Dawar da hasken fage daga hoto guda ɗaya matsala ce ta zamani, marar tsari a cikin hangen nesa na kwamfuta. Hanyoyin gargajiya, musamman don fage na cikin gida, sau da yawa sun dogara da taswirorin muhalli—ƙa'idar haske mai nisa da yawanci ke cin zarafi ta hanyar hanyoyin haske na gida kamar fitilu, wanda ke haifar da sakamako marar gaskiya don ayyuka kamar shigar da abubuwa na kama-da-wane (duba Hoto na 1). Wannan takarda ta gabatar da sabuwar hanyar koyon zurfi wacce ke ƙetare wannan iyaka ta hanyar ƙididdigar samfurin haske na 3D mai siffa kai tsaye daga hoton cikin gida mai ƙarancin ƙarfi (LDR).

Babban gudummawar ita ce canji daga wakilcin duniya, wanda ya dogara da jagora, zuwa saitin hanyoyin haske na 3D masu rabuwa tare da sifofi na geometric (matsayi, yanki) da na hotometric (ƙarfi, launi). Wannan yana ba da damar haske mai bambancin wuri, ma'ana inuwa da inuwa sun daidaita daidai da wurin abu a cikin fage, kamar yadda aka nuna a cikin hoton teaser.

2. Hanyoyin Bincike

2.1 Wakilin Haske Mai Siffa

Hanyar tana wakiltar hasken cikin gida a matsayin tarin fitilu na yanki $N$. Kowane haske $L_i$ an ƙayyade shi ta hanyar:

• Matsayi: $\mathbf{p}_i \in \mathbb{R}^3$ (wurin 3D a cikin ma'auni na fage).
• Yanki: $a_i \in \mathbb{R}^+$ (yana bayyana girman sararin haske).
• Ƙarfi: $I_i \in \mathbb{R}^+$.
• Launi: $\mathbf{c}_i \in \mathbb{R}^3$ (ƙimar RGB).

Wannan saitin sifofi $\Theta = \{ \mathbf{p}_i, a_i, I_i, \mathbf{c}_i \}_{i=1}^{N}$ yana ba da taƙaitaccen bayanin hasken fage mai fassara ta zahiri wanda za'a iya kimanta shi a kowane wuri na 3D.

2.2 Tsarin Cibiyar Sadarwa

An horar da cibiyar sadarwar jijiya mai zurfi don dawo da sifofi $\Theta$ daga shigarwar RGB guda ɗaya. Cibiyar sadarwa tana bin tsarin mai ɓoyewa-mai bayyanawa:

1. Mai ɓoyewa: Gindin convolutional (misali, ResNet) yana cire siffar sifa daga hoton shigarwa.
2. Mai bayyanawa: Layer masu cikakken haɗin kai suna zana siffar ɓoyewa zuwa sifofi na fitarwa $N \times 8$ (3 don matsayi, 1 don yanki, 1 don ƙarfi, 3 don launi).

An horar da samfurin akan bayanan taswirorin muhalli na High Dynamic Range (HDR) na cikin gida, waɗanda aka yiwa alama da hannu tare da taswirorin zurfi masu dacewa da fitilu masu siffa.

2.3 Layer na Zane Mai Banbanta

Wani sabon abu shine layer mai banbanta wanda ke canza sifofin da aka annabta $\Theta$ zuwa taswirar muhalli na yau da kullun $E(\Theta)$ a takamaiman wurin tambaya. Wannan yana ba da damar lalacewa a cikin yankin hoto (kwatanta zane da taswirar muhalli na gaskiya) ba tare da buƙatar daidaitattun alaƙa tsakanin kowane annabta da fitilu na gaskiya ba. Ana iya tsara aikin asara kamar haka:

$\mathcal{L} = \| E(\Theta) - E_{gt} \| + \lambda \mathcal{R}(\Theta)$

inda $E_{gt}$ shine taswirar muhalli na gaskiya, kuma $\mathcal{R}$ shine sharadi na ƙayyadaddun sharadi akan sifofi.

3. Gwaje-gwaje & Sakamako

3.1 Ƙididdiga ta Ƙididdiga

Takardar tana kimanta aikin ta amfani da ma'auni na yau da kullun don ƙididdigar haske, kamar Matsakaicin Kuskuren Angular (MAE) akan taswirorin muhalli da aka annabta da ma'auni na fahimta. Hanyar siffa da aka gabatar tana nuna matsakaicin aikin ƙididdiga idan aka kwatanta da tsoffin hanyoyin da ba na siffa ba (tsinkayar taswirar muhalli) kamar Gardner et al. [7], musamman lokacin da ake kimanta daidaiton haske a wurare da yawa a cikin fage.

3.2 Ƙididdiga ta Halaye

Sakamako na halaye yana nuna fa'ida bayyananne. Fitilun da aka annabta sun dace da hanyoyin haske na gaskiya a cikin hoton shigarwa (tagogi, fitilu). Lokacin da aka hango su, taswirorin muhalli da aka sake ginawa suna nuna cikakkun bayanai mafi daidai (inuwa mai kaifi) da sake haifar da launi idan aka kwatanta da sakamako masu ɓarna, matsakaici daga hanyoyin duniya.

3.3 Haɗa Abubuwa na Kama-da-wane

Aikace-aikacen da ya fi jan hankali shine shigar da abu na kama-da-wane na hoto. Ta amfani da sifofin haske na 3D da aka ƙididdige, ana iya zana abu na kama-da-wane tare da daidaitaccen, inuwa da inuwa mai bambancin wuri. Yayin da wani abu ke motsawa ta cikin fage (misali, daga tebur zuwa ƙarƙashin fitila), haskensa yana canzawa da gaske—wani aiki wanda ba zai yiwu ba tare da taswirar muhalli guda ɗaya ta duniya ba. Hoto na 1(b) a cikin PDF yana kwatanta wannan tare da jagororin inuwa daban-daban da ƙarfin inuwa don sanya abubuwa daban-daban.

4. Bincike na Fasaha & Tsarin Aiki

4.1 Fahimta ta Asali & Tsarin Ma'ana

Bari mu yanke fuskar ilimi. Babban fahimta a nan ba wani ƙarin ci gaba ne kawai a cikin tsarin cibiyar sadarwa ba; yana da sake tattara asali na bayanin matsala. Marubutan sun gane cewa "taswirar muhalli" na yau da kullun na aikin da ya gabata (kamar aikin Gardner et al.) a zahiri ya kasance ƙarshen matattu don aikace-aikacen AR/VR na gaske. Yana da wayo mai haske wanda ke magance alamar (tsinkayar haske) amma yana watsi da cuta (haske na gida ne). Tsarin ma'anarsu yana da kaifi sosai: 1) Amince da ƙuntatawa ta zahiri (fitilu na cikin gida), 2) Zaɓi wakilcin da ke ƙunshe da samfurinsa (fitilu na 3D mai siffa), 3) Gina gada (mai zane mai banbanta) don har yanzu yin amfani da yalwar bayanan hoto don horo. Wannan yana tunawa da canji a cikin samfuran samarwa daga tsinkayar pixel kai tsaye (kamar GANs na farko) zuwa koyon wakilcin ɓoyayyen tsarin 3D, kamar yadda aka gani a cikin tsarin aiki kamar NeRF.

4.2 Ƙarfafawa & Kurakurai

Ƙarfafawa:

• Daidaiton Zahiri & Gyara: Saitin sifofi mafarki ne na mai zane. Kuna iya gyara matsayin haske ko ƙarfi kai tsaye—matakin sarrafa da ba ya nan daga pixels na taswirar muhalli ba. Wannan yana haɗa tazarar tsakanin ƙididdigar AI da hanyoyin zane na aiki.
• Sanin Sarari: Wannan shine fasalin mai kisa. Yana magance "ɗayan haske ya dace da kowa" na hanyoyin da suka gabata, yana sa haɗa gaskiyar ƙarin gaskiyar gaskiya ya yiwu.
• Wakilcin Bayanai Mai Ƙarfi: ƴan sifofi da yawa sun fi cikakken taswirar muhalli na HDR taƙaitacce, wanda zai iya haifar da ƙarin ƙwararrun koyo daga ƙarancin bayanai.

Kurakurai & Tambayoyi Budadde:

• Matsalar "N": Cibiyar sadarwa tana annabta ƙayyadaddun adadin fitilu, wanda aka riga aka ayyana. Me game da fage masu ƙarin tushe ko ƙasa? Wannan ƙa'idar ce mai rauni. Cibiyoyin sadarwar zane masu ƙarfi ko hanyoyin da aka yi wahayi zuwa gano abu na iya zama matakai masu zuwa.
• Dogaro da Geometry: Horon hanyar da kimantawa sun dogara da bayanan da aka yiwa alama da zurfi. Ayyukanta a cikin daji, ba tare da sanannen geometry ba, babbar tambaya ce da ba a amsa ba. Yana iya haɗa matsalolin ƙididdigar haske da geometry sosai.
• Rufe & Haɗin kai Mai Sarƙaƙiya: Samfurin na yanzu yana amfani da fitilu na yanki masu sauƙi. Haske na cikin gida na gaske ya ƙunshi haɗin kai masu sarƙaƙiya, rufewa, da saman da ba su da yawa (misali, teburori masu sheki). Sakamakon haɗawa na takardar, duk da yake yana da kyau, har yanzu yana da ɗan "tsaftataccen" kamannin CG wanda ke nuna waɗannan rikitattun abubuwan da suka ɓace.

4.3 Fahimta Mai Aiki

Ga masu aiki da masu bincike:

1. Benchmarking Yana Da Muhimmanci: Kar ku kawai ba da rahoton kuskuren angular akan taswirar muhalli da aka yanke. Dole ne fannin ya karɓi ma'auni na tushen aiki kamar maki na gaskiya a cikin ayyukan haɗa abubuwa, wanda binciken ɗan adam ko ƙwararrun samfuran fahimta suka yanke hukunci (misali, bisa LPIPS ko makamantansu). Hotunan haɗawa na halaye na wannan takarda sun fi gamsarwa fiye da kowane ma'auni na lamba ɗaya.
2. Karɓi Kimiyyar Lissafi Mai Banbanta: Mai zane mai banbanta shine maɓalli. Wannan yanayin, wanda aka shahara da ayyuka kamar PyTorch3D da Mitsuba 2, shine makoma don haɗa koyo da zane. Ku saka hannun jari a cikin gina waɗannan Layer don yankinku.
3. Duba Bayan Kulawa: Bukatar haɗin taswirorin muhalli na HDR tare da zurfi shine maƙulli. Ci gaba na gaba zai zo daga hanyoyin da ke koyon fifikon haske daga hotunan intanet ko bidiyo marasa lakabi, watakila ta amfani da ƙuntatawa na kai daga geometry mai ra'ayi da yawa ko daidaiton abu, kamar ƙa'idodi a cikin ayyukan abin tunawa kamar "Koyon Gani a cikin Duhu" ko daga bayanai kamar MegaDepth.

Misalin Tsarin Bincike (Ba Code ba): Don kimanta kowane sabuwar takarda ta ƙididdigar haske da gaske, yi amfani da wannan tsarin aiki mai maki uku: 1) Amincin Wakilci: Shin tsarin fitarwa yana goyan bayan bambancin sarari da gyara ta zahiri? (Mai Siffa > Taswirar Muhalli). 2) Horo na Gaskiya: Shin hanyar tana buƙatar kulawa cikakke marar yiwuwa (cikakken binciken fage na 3D) ko tana iya koyo daga sigina masu rauni? 3) Aikin Aiki: Shin yana nuna inganta aikace-aikacen gaske (haɗawa, sake haskakawa) fiye da ma'auni na roba? Wannan takarda tana da maki sosai akan 1 da 3, amma 2 ya kasance kalubale.

5. Ayyuka na Gaba & Jagorori

Tasirin ƙididdigar haske mai siffa mai ƙarfi yana da yawa:

• Ƙarin Gaskiya & Gaskiya ta Kama-da-wane: Ba da damar abun ciki na AR na gaske mai dorewa wanda ke hulɗa da gaskiya tare da hasken ɗaki. Abubuwa na kama-da-wane za su iya jefa inuwa daidai akan saman gaske kuma su bayyana an haskaka su ta fitilar teburin mai amfani.
• Daukar Hoto na Lissafi & Gyara Bayan Daukar Hoto: Ba da damar gyaran hoto na matakin ƙwararru kamar sake haskakawa bayan ɗaukar hoto, shigar da abu, da daidaita inuwa a cikin hotuna da bidiyoyi.
• Hangen Nesa na Gine-gine & Ɗaukar Hoton Cikin Gida: Masu amfani za su iya ɗaukar hoton ɗaki kuma a zahiri "gwada" kayan haske daban-daban ko kayan ɗaki a ƙarƙashin yanayin haske na yanzu.
• Robotics & AI mai jiki: Samar da robots tare da ƙarin fahimtar muhallin 3D, taimakawa wajen kewayawa, sarrafawa, da fahimtar fage.

Jagororin Bincike na Gaba:

1. Haɗin Ƙididdiga tare da Geometry: Haɓaka samfuran ƙarshe-zuwa-ƙarshe waɗanda ke ƙididdige zurfin fage, shimfidawa, da haske daga hoto guda ɗaya, rage dogaro da geometry da aka riga aka lissafa.
2. Ƙididdiga Mai Ƙarfi & Na Bidiyo: Tsawaita hanyar zuwa bidiyo don ƙididdigar canje-canjen lokaci a cikin haske (misali, wani yana kunna fitila/kashewa).
3. Haɗawa tare da Zane na Jijiya: Haɗa fitilu masu siffa tare da filayen haske na jijiya (NeRFs) don cimma sabon ra'ayi na ra'ayi na gaske da gyara.
4. Koyo marar Kulawa & Mai Rauni: Bincika koyo daga tarin hotuna a cikin daji ba tare da gaskiyar HDR/zurfi ba.

6. Nassoshi

1. Gardner, M.-A., Hold-Geoffroy, Y., Sunkavalli, K., Gagné, C., & Lalonde, J.-F. (2019). Deep Parametric Indoor Lighting Estimation. arXiv preprint arXiv:1910.08812.
2. Gardner, M.-A., et al. (2017). Learning to Predict Indoor Illumination from a Single Image. ACM TOG.
3. Debevec, P. (1998). Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography. ACM SIGGRAPH.
4. Hold-Geoffroy, Y., Sunkavalli, K., et al. (2017). Deep Outdoor Illumination Estimation. IEEE CVPR.
5. Mildenhall, B., et al. (2020). NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV.
6. Zhang, R., et al. (2018). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. IEEE CVPR. (LPIPS)
7. Li, Z., & Snavely, N. (2018). MegaDepth: Learning Single-View Depth Prediction from Internet Photos. IEEE CVPR.