UniLight: Wani Haɗaɗɗen Wakilcin Hasken Multimodal don Hangar Nesa da Zane-zane

1. Gabatarwa & Bayyani

Hasken wani muhimmin sashi ne amma mai sarƙaƙƙiya a cikin bayyanar gani a cikin hangar nesa da zane-zane. Wakilcin gargajiya—taswirorin muhalli, taswirorin hasken haske, harmonics na siffar duniya, da bayanin rubutu—sun kasance ba su dace ba sosai, suna haifar da manyan cikas ga fahimtar haske da sarrafa shi tsakanin nau'ikan watsa labarai. UniLight yana magance wannan rarrabuwar ta hanyar gabatar da wani haɗaɗɗen sararin haɗin kai wanda ke haɗa waɗannan nau'ikan watsa labarai daban-daban.

Babban ƙirƙira ya ta'allaka ne a cikin horar da masu rufe rubutu na musamman (don rubutu, hotuna, hasken haske, da taswirorin muhalli) ta amfani da tsarin koyon kwatance, yana tilasta wakilcinsu su daidaita a cikin wani sarari mai girma mai girma. Wani aiki na taimako wanda ke hasashen ƙididdiga na harmonics na siffar duniya yana ƙarfafa fahimtar model game da kaddarorin haske na shugabanci.

Mahimman Fahimta

Haɗin kai: Ya ƙirƙiri wakilci guda ɗaya, mai daidaituwa daga tsarin hasken da ba su dace ba a baya.
Sassauci: Yana ba da damar sabbin aikace-aikace kamar ma'ajin bayanai tsakanin nau'ikan watsa labarai da samarwa bisa sharadi.
Dogaro da Bayanai: Yana amfani da bututun bayanai mai yawa mai ma'auni don horarwa.

2. Tsarin Aiki na Asali

Tsarin UniLight an tsara shi don cire da daidaita bayanin haske daga tushe da yawa zuwa cikin sararin haɗakarwa gama gari.

2.1 Tsarin Sararin Haɗin Kai

Model ɗin ya kafa wani sarari na haɗin kai $\mathcal{Z} \subset \mathbb{R}^d$, inda $d$ shine girman haɗakarwa. Kowane nau'in shigarwa $x_m$ (inda $m \in \{\text{rubutu, hoto, hasken haske, taswirar muhalli}\}$) ana sarrafa shi ta hanyar mai rufe rubutu na musamman $E_m$ don samar da haɗakarwa $z_m = E_m(x_m) \in \mathcal{Z}$. Manufar ita ce tabbatar da cewa $z_m$ don nau'ikan watsa labarai daban-daban, lokacin da suke bayyana yanayin haske iri ɗaya, suna da alaƙa sosai.

2.2 Masu Rufe Rubutu na Musamman

Mai Rufe Rubutu: Ya dogara ne akan tsarin transformer (misali, mai rufe rubutu irin na CLIP) don sarrafa bayanin harshe na halitta kamar "waje, haske mai haske kai tsaye daga sama dama."
Mai Rufe Hotuna/Taswirar Muhalli/Hasken Hasken: Suna amfani da Vision Transformers (ViTs) don sarrafa wakilcin gani na haske (taswirorin muhalli na HDR, taswirorin hasken haske, ko hotuna gabaɗaya).

2.3 Manufofin Horarwa

Horarwa ya haɗu da manyan manufofi guda biyu:

Asarar Kwatance ($\mathcal{L}_{cont}$): Yana amfani da ƙididdiga mai kwatance da hayaniya (misali, InfoNCE) don jawo haɗakarwa na yanayin haske iri ɗaya daga nau'ikan watsa labarai daban-daban (ma'aurata masu kyau) da kawar da haɗakarwa daga al'amura daban-daban (ma'aurata marasa kyau). Don guntu na $N$ ma'aurata masu nau'ikan watsa labarai da yawa, asarar don anga $i$ ita ce: $$\mathcal{L}_{cont}^{i} = -\log\frac{\exp(\text{sim}(z_i, z_{i}^+) / \tau)}{\sum_{j=1, j\neq i}^{N} \exp(\text{sim}(z_i, z_j) / \tau)}$$ inda $\text{sim}$ shine kamancen cosine kuma $\tau$ shine sigar zafin jiki.
Asarar Taimako na Harmonics na Siffar Duniya ($\mathcal{L}_{sh}$): Kan multi-layer perceptron (MLP) yana hasashen ƙididdiga na wakilcin harmonics na siffar duniya na digiri na 3 (SH) daga haɗakar haɗin kai $z$. Wannan asarar koma baya $\mathcal{L}_{sh} = ||\hat{Y} - Y||_2^2$ tana tilasta rufewa na bayanin haske na shugabanci, mai mahimmanci ga ayyuka kamar sake haskakawa.

Jimlar asara ita ce $\mathcal{L} = \mathcal{L}_{cont} + \lambda \mathcal{L}_{sh}$, inda $\lambda$ ke daidaita sharuɗɗan biyu.

3. Aiwatar da Fasaha

3.1 Tsarin Lissafi

Hasashen harmonics na siffar duniya yana da mahimmanci don kama shugabanci. Harmonics na siffar duniya $Y_l^m(\theta, \phi)$ suna samar da tushe mai daidaituwa a kan duniya. Ana iya kusantar haske kamar haka: $$L(\theta, \phi) \approx \sum_{l=0}^{L}\sum_{m=-l}^{l} c_l^m Y_l^m(\theta, \phi)$$ inda $L$ shine iyakar band (digiri 3 a cikin UniLight), kuma $c_l^m$ su ne ƙididdiga na SH. Aikin taimako yana koyon taswira $f: \mathcal{Z} \rightarrow \mathbb{C}^{16}$ (don ƙididdiga na gaske $c_l^m$ har zuwa $l=3$).

3.2 Bututun Bayanai

Bututun nau'ikan watsa labarai da yawa ya fara ne daga ainihin bayanan taswirorin muhalli na HDR. Daga cikin waɗannan, ana samar da taswirorin hasken haske na roba, kuma ana samo bayanan rubutu masu dacewa ko dai daga metadata ko kuma ana samar da su ta amfani da model na harshe na hangen nesa. Wannan bututun yana ba da damar ƙirƙirar bayanan horarwa masu nau'ikan watsa labarai da yawa masu girma daga nau'in tushe guda ɗaya.

4. Sakamakon Gwaji

An kimanta UniLight akan ayyuka guda uku masu zuwa, yana nuna amfanin wakilcinsa na haɗin kai.

4.1 Ma'ajin Bayanai dangane da Hasken

Aiki: An ba da tambaya a cikin nau'in watsa labarai ɗaya (misali, rubutu), samo mafi kama misalan haske daga ma'ajin bayanai na wani nau'in watsa labarai (misali, taswirorin muhalli).
Sakamako: UniLight ya fi manyan hanyoyin tushe waɗanda ke amfani da fasali na musamman na nau'in watsa labarai. Haɗakar haɗin kai ya ba da damar bincike mai ma'ana na kamance tsakanin nau'ikan watsa labarai, kamar gano taswirar muhalli da ta dace da "shuɗin sama, na halitta" daga rubutu.

4.2 Samar da Taswirar Muhalli

Aiki: Sanya sharadi akan model ɗin samarwa (misali, model ɗin yaduwa) akan haɗakar UniLight daga kowane nau'in shigarwa don haɗa sabon taswirar muhalli na HDR mai ƙima.
Sakamako: Taswirorin da aka samar sun kasance masu kama da hoto kuma sun dace da ma'ana tare da shigarwar sharadi (rubutu, hoto, ko hasken haske). Model ɗin ya yi nasara wajen kama kaddarorin haske na duniya kamar shugabanci na rana da launin sama.

4.3 Sarrafa Haɗakar Hotuna ta hanyar Yaduwa

Aiki: Yi amfani da haɗakar UniLight don jagorantar haske a cikin model ɗin yaduwa daga rubutu zuwa hoto, yana ba da damar sarrafa haske a fili daban da bayanin abun ciki.
Sakamako: Ta hanyar shigar da haɗakar haske cikin tsarin yaduwa (misali, ta hanyar kulawa ta giciye ko sassan adafta), masu amfani za su iya samar da hotuna tare da takamaiman haske mai sarrafawa wanda aka bayyana ta rubutu ko hoton tunani, wani ci gaba mai mahimmanci akan sarrafawa na gaba ɗaya na gaggawa.

Taƙaitaccen Aiki

Daidaiton Ma'ajin Bayanai (Top-1): ~15-25% mafi girma fiye da manyan hanyoyin tushe na musamman na nau'in watsa labarai.
Makin FID na Samuwa: An inganta da ~10% idan aka kwatanta da model ɗin da ba su da asarar taimako na SH.
Zaɓin Mai Amfani (Sarrafa Hasken): >70% fifiko ga hotunan jagorancin UniLight fiye da sakamakon yaduwa na tushe.

5. Tsarin Bincike & Nazarin Lamari

Aiwatar da Tsarin: Don bincika hanyar kimanta haske, zamu iya amfani da tsarin da ke kimanta Ƙarfin Wakilci, Sassaucin Tsakanin Nau'ikan Watsa Labarai, da Ingancin Aiki mai Zuwa.

Nazarin Lamari - Hotunan Samfur na Kama-da-wane:

Manufa: Saka model 3D na takalmi a cikin haske da ya dace da hoton faɗuwar rana da mai amfani ya ɗora.
Tsari tare da UniLight:
- Hoton tunanin mai amfani an rufe shi ta hanyar mai rufe hoto zuwa cikin sararin haɗin kai $\mathcal{Z}$.
- An samo wannan haɗakar haske $z_{img}$.
- Zaɓi A (Ma'ajin Bayanai): Nemo mafi kama taswirar muhalli na HDR da aka riga aka samu daga ɗakin karatu don amfani a cikin mai saka.
- Zaɓi B (Samuwa): Yi amfani da $z_{img}$ don sanya sharadi akan mai samarwa, ƙirƙirar sabon taswirar muhalli na HDR mai inganci wanda aka keɓance ga ainihin launukan faɗuwar rana.
Sakamako: An saka takalmin 3D tare da haske wanda ya dace da haske mai zafi, na shugabanci na hoton faɗuwar rana, yana ba da damar daidaiton alamar kasuwanci da sarrafa kyan gani a cikin kayan talla.

Wannan yana nuna ƙimar aiki ta UniLight wajen haɗa tazarar tsakanin shigarwar mai amfani na yau da kullun (hoton wayar hannu) da bututun zane-zane na ƙwararru.

6. Bincike Mai Zurfi & Ra'ayoyin Kwararru

Fahimta ta Asali: UniLight ba wani mai kimanta haske kawai ba ne; yana da tushen harshen tsaka-tsaki don haskakawa. Babban ci gaban shine kula da haske a matsayin ra'ayi na farko, wanda bai dace da nau'in watsa labarai ba, kamar yadda CLIP ta ƙirƙiri sarari na haɗin kai don hotuna da rubutu. Wannan sake tsarawa daga kimantawa zuwa fassara shine abin da ke buɗe sassaucinsa.

Kwararar Hankali & Matsayin Dabarun: Takardar ta gano daidai rarrabuwar a cikin fagen—hasumiya na Babel inda harmonics na siffar duniya ba za su iya magana da gaggawar rubutu ba. Maganinsu ya bi tsarin da aka tabbatar: koyon kwatance don daidaitawa, wanda ayyuka kamar SimCLR da CLIP suka yada, da kuma mai daidaita yanki na musamman (hasashen SH). Wannan fasaha ce mai wayo, ba bincike na gaba ɗaya na sama ba. Yana sanya UniLight a matsayin matsakaiciyar kayan aiki da ake buƙata tsakanin duniyar AI mai samarwa (wanda ke buƙatar sarrafawa) da takamaiman buƙatun bututun zane-zane (wanda ke buƙatar sigogi).

Ƙarfi & Kurakurai:

Ƙarfi: Bututun bayanai mai nau'ikan watsa labarai da yawa babban kadari ne, yana mai da matsalar ƙarancin zuwa fa'ida mai ma'auni. Zaɓin hasashen SH a matsayin aikin taimako yana da kyau—yana shigar da mahimman ilimin farko na zahiri (shugabanci) cikin wani haɗakarwa na gaba ɗaya na dogaro da bayanai.
Kurakurai & Gaps: Takardar tana shiru sosai akan haske mai canzawa ta sarari. Yawancin al'amuran duniya na gaske suna da inuwa masu sarƙaƙƙiya da tushen haske na gida. Shin haɗakarwa guda ɗaya ta duniya daga mai rufe hoto zata iya kama hakan? Da alama a'a. Wannan yana iyakance amfani ga al'amuran da ba na Lambertian ba ko na ciki mai sarƙaƙiya. Bugu da ƙari, yayin da yake amfani da model ɗin yaduwa don samarwa, ƙaƙƙarfan haɗin gwiwa ba a bayyane ba. Shin sharadi ne mai sauƙi, ko sarrafawa mai zurfi kamar ControlNet? Rashin cikakken bayanin gine-gine a nan wata dama ce da aka rasa don sake yin samfur.

Idan aka kwatanta da hanyoyin haske na ƙa'idar NeRF (kamar NeILF), UniLight yana da amfani sosai don gyara amma ba daidai ba a zahiri. Yana cin wasu daidaito don amfani da sauri—daidaitaccen sulhu don aikace-aikace da yawa.

Fahimta Mai Aiki:

Ga Masu Bincike: Babban kofa da ba a buɗe ba a nan shine faɗaɗa ra'ayin "wakilci na haɗin kai" zuwa lokaci (jerin haske don bidiyo) da sarari (haɗakarwa kowane pixel ko kowane abu). Mataki na gaba shine "UniLight++" wanda ke sarrafa cikakken sarƙaƙiyar ma'auni na jigilar haske, ba kawai haske mai nisa ba.

Ga Masu Aiki (Shugabannin Fasaha, Manajoji na Samfur): Wannan yana shirye don haɗin kai na matukin jirgi a cikin kayan aikin ƙirƙirar abun ciki na dijital. Amfanin nan take shine a cikin zane-zane na ra'ayi da riga-kafi: ba da damar masu fasaha su bincika ɗakunan karatu na haske da rubutu ko hotuna, ko kuma su yi gaggawar yin ƙirar al'amura tare da daidaitaccen haske daga allon yanayi. Ba da fifiko ga haɗin kai tare da injuna kamar Unity ko Unreal ta hanyar kayan haɗi wanda ke canza haɗakar UniLight zuwa binciken haske na asali.

Ga Masu Zuba Jari: Ku yi fare akan kamfanoni waɗanda ke gina "gatura da magarya" don AI mai samarwa a fagagen ƙirƙira. UniLight yana misalta irin fasahar abubuwan more rayuwa—ba da damar sarrafawa mafi kyau—wanda zai zama mahimmanci yayin da samfuran samarwa ke motsawa daga sabon abu zuwa kayan aikin samarwa. Kasuwar bayanan haske da kayan aiki tana daɗe don rugujewa.

A ƙarshe, UniLight wani muhimmin mataki ne mai amfani na gaba. Ba ya warware haske, amma yana warware matsalar sadarwa game da haske, wanda ya kasance babban cikas. Nasararsa za a auna shi da saurin da za a yi amfani da shi a cikin daidaitaccen kayan aikin masu fasaha da masu haɓakawa.

7. Ayyukan Gaba & Hanyoyi

Ƙarfafa Gaskiya & Gaskiya ta Kama-da-wane (AR/VR): Kimanta hasken muhalli na ainihin lokaci daga ciyarwar kyamarar wayar hannu (nau'in hoto) don haskaka abubuwa na kama-da-wane da aka sanya su cikin muhallin mai amfani.

Ƙirƙirar Abun Ciki ta Atomatik: Haɗin kai cikin bututun samar da fim da wasan kwaikwayo don saitin haske ta atomatik dangane da bayanin kula na darekta (rubutu) ko hotunan fim na tunani (hoto).

Hangen Gine-gine & Ɗaukar Hotuna na Ciki: Ba da damar abokan ciniki su bayyana yanayin haske da ake so ("zauren maraice mai dadi") da kuma ganin samfuran gine-gine na 3D a ƙarƙashin wannan haske nan take.

Saka Hotuna ta Jijiya & Juyin Juya Hali na Zane-zane: Yin aiki a matsayin fifikon haske mai ƙarfi don ayyukan juyin juya hali, yana taimakawa wajen raba lissafi, kayan aiki, da haske daga hotuna guda ɗaya yadda ya kamata.

Hanyar Bincike - Hasken Mai Ƙarfi: Faɗaɗa tsarin don ƙirar canje-canjen haske akan lokaci don sake haskaka bidiyo da gyara.

Hanyar Bincike - Hasken Na Musamman: Koyon zaɓin haske na musamman na mai amfani daga bayanan hulɗa da kuma amfani da su a cikin abun ciki da aka samar ko aka gyara.

8. Nassoshi

Zhang, Z., Georgiev, I., Fischer, M., Hold-Geoffroy, Y., Lalonde, J-F., & Deschaintre, V. (2025). UniLight: A Unified Representation for Lighting. arXiv preprint arXiv:2512.04267.

Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020). NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV.

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., et al. (2021). Learning Transferable Visual Models From Natural Language Supervision. ICML (CLIP).

Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A Simple Framework for Contrastive Learning of Visual Representations. ICML (SimCLR).

Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. CVPR.

Ramamoorthi, R., & Hanrahan, P. (2001). An Efficient Representation for Irradiance Environment Maps. SIGGRAPH (Spherical Harmonics for Lighting).