NieR: Yin Haske na Yanayin Yin Haske bisa ga Tsarin Yanayin Yanayin - Binciken Fasaha

1. Gabatarwa & Bayyani

NieR (Yin Haske na Yanayin Yin Haske bisa ga Tsarin Yanayin Yanayin) sabon tsari ne da aka ƙera don magance babban ƙalubalen yin haske da kayan aiki na gaske a cikin yanayi na 3D mai motsi, musamman a cikin simintin tuƙi mai sarrafa kansa. Hanyoyin gargajiya na 3D Gaussian Splatting, duk da cewa suna da inganci, sau da yawa sun kasa yin ƙirar hulɗar haske da saman da suka yi rikitarwa daidai, musamman hasken da ke nunawa akan kayan kamar fentin mota, wanda ke haifar da kurakurai na gani kamar shuɗewa da wuce gona da iri. NieR ya gabatar da hanyar aiki mai fuska biyu: Sashen Rarraba Haske (LD) wanda ke raba gudummawar haske ta amfani da tsarin yanayin saman, da kuma Sashen Haɓaka Matsakaicin Yanayin Yanayin a Matakai (HNGD) wanda ke ƙara yawan Gaussian a wuraren da ke da rikitarwar lissafi da bambancin haske bisa ga yanayi. Wannan haɗin yana nufin haɓaka ingancin yin haske sosai ga abubuwa masu haske a ƙarƙashin hasken muhalli mai motsi.

2. Hanyoyin Aiki

Babban ƙirƙira na NieR ya ta'allaka ne a haɗa ƙa'idodin yin haske na zahiri cikin tsarin 3D Gaussian Splatting.

3. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Tsarin ya ginu bisa ga tushen 3D Gaussian Splatting. Kowane Gaussian an ƙara shi da sifofi don ma'aunin haske $k_s$ da ingantaccen vector na yanayin yanayi. Lissafin sashen LD an haɗa shi cikin na'urar rasterizer mai tushen tile. Sashen HNGD yana aiki yayin matakin sarrafa yawan adadin daidaitawa na madauki, yana amfani da bayanan yanayin da aka adana kowane Gaussian don lissafin matakan gida da kuma haifar da haɓaka kafin zagaye na gaba.

Haɗin Ma'auni Mai Muhimmanci: Launi $C$ na pixel a cikin haɗin splatting na ƙarshe yanzu aiki ne na hasken da aka rarraba:

$C = \sum_{i \in \mathcal{N}} c_i \cdot \alpha_i \prod_{j=1}^{i-1}(1-\alpha_j)$

inda $c_i$ yanzu an samo shi daga $L_o^i$ (hasken da aka rarraba na Gaussian na i-th) maimakon siffar RGB mai sauƙi.

4. Sakamakon Gwaji & Aiki

Takardar tana kimanta NieR akan bayanan da ke nuna abubuwa masu ƙalubalantar haske (misali, motoci) a cikin yanayin hanya. Sakamako na halitta yana nuna raguwar shuɗewa da karkace a kan jikin mota da tagogi idan aka kwatanta da vanilla 3DGS da sauran hanyoyin SOTA kamar Instant-NGP da Plenoxels. Hasken ya fi ƙunshe da gaske, yana guje wa tasirin "fure".

Ma'auni na ƙididdiga (PSNR, SSIM, LPIPS) da aka ruwaito akan ma'auni na yau da kullun (mai yuwuwa na roba ko yanayin tuƙi da aka ɗauka) sun nuna aiki mafi girma. Babban jadawali zai kwatanta PSNR a cikin hanyoyi akan jerin abubuwa tare da tushen haske mai motsi, yana nuna kwanciyar hankali na NieR. Wani zane zai kwatanta rarraba Gaussian kafin da bayan HNGD, yana nuna ƙara yawan adadin a kusa da siffar mota da yankunan haske.

5. Tsarin Bincike & Nazarin Lamari

Nazarin Lamari: Yin Haske na Titin Danye da Dare

Wannan yanayin ya haɗu da kwalta mai watsawa, tafkunan ruwa masu haske sosai, da fitilun mota masu motsi. Daidaitaccen ƙirar 3DGS zai yi wahala: tafkunan na iya bayyana a shuɗe ko kuma rashin kaifin haske, canjin launi na fitilu. Tsarin NieR zai sarrafa shi kamar haka:

1. Sashen LD: Ga Gaussian akan tafki, ana koyon babban $k_s$. $L_s$ yana ɗaukar hasken kai tsaye, kamar madubi na fitilar mota (launi, ƙarfi). $L_d$ yana ɗaukar ƙaramin hasken birni na yanayi akan saman da aka jika.
2. Sashen HNGD: Iyaka tsakanin titin bushe (ƙananan matakan yanayin yanayi) da tafki (babban matakan yanayin yanayi saboda katsewar saman) yana haifar da haɓaka. An ba da ƙarin Gaussians don ƙirar gefen haske daidai.
3. Sakamako: Yin haske na ƙarshe yana nuna haske mai haske, mai haske na fitilar mota a cikin tafki, wanda aka haɗa shi da duhu, titin da ke watsawa, yana haɓaka gaske na gaske na yanayi kuma yana da mahimmanci ga algorithms na zurfi/fahimta a cikin tuƙi mai sarrafa kansa.

6. Bincike Mai Zurfi & Fassarar Kwararru

Mahimmin Fahimta: NieR ba kawai ƙaramin gyara ba ne; yana juyawa daga dabarar kallon Gaussians a matsayin kawai ƙullun bayyanar zuwa kula da su a matsayin binciken haske na ƙananan lissafi. Ta hanyar saka ƙirar PBR mai sauƙi (LD) da ƙa'idar ingantawa mai hankali ga lissafi (HNGD), yana kai hari kai tsaye ga rashin daidaiton asali tsakanin sifofin Gaussians masu santsi, na ƙididdiga da kuma yanayin haske mai haske wanda ke motsa jiki. Wannan shine mabuɗin buɗewa ga kayan kamar ƙarfe da gilashi a cikin yin haske na ainihin lokaci.

Kwararren Tsari: Hankali yana da kyau. Matsala: Gaussians ba su da kyau a haske mai kaifi. Tushen Dalili 1: Suna haɗa hasken watsawa/haske. Magani: Rarraba haske (LD). Tushen Dalili 2: Suna da yawa a wuraren da haske ke faruwa. Magani: Haɓaka inda lissafi/haske ke canzawa da sauri (HNGD). Amfani da matakan yanayin yanayi a matsayin siginar haɓaka yana da wayo—yana wakiltar mahimmanci na gani wanda ya fi kwanciyar hankali fiye da matakan launi kawai.

Ƙarfi & Kurakurai:

• Ƙarfi: Haɗin yana da sauƙi, yana adana aikin ainihin lokaci. Mayar da hankali kan tuƙi mai sarrafa kansa yana da hikima ta kasuwanci. Hanyar tana dacewa da sauran ingantattun 3DGS.
• Kurakurai: Takardar ta nuna amma ba ta magance cikakken hasken tsakanin juna da zubar da launi ba—rauni da aka sani na yawancin hanyoyin yin haske na jijiyoyi. Ana koyon sigar $k_s$ kowane Gaussian, wanda bazai yi daidai da kayan da ba a gani ba sosai. Idan aka kwatanta da cikakkun hanyoyin PBR na tushen NeRF (kamar NeRF-OSR), yana ciniki: yana da sauri amma yana iya zama ƙasa da daidaiton jiki don haske na duniya mai rikitarwa.

Fahimta Mai Aiki:

1. Ga Masu Bincike: Haɗin LD/HNGD samfuri ne. Bincika maye gurbin sauƙi na BRDF a cikin LD tare da ƙaramin MLP don ƙarin kayan aiki masu rikitarwa. Bincika amfani da HNGD don wasu sifofi kamar lakabin ma'ana.
2. Ga Masu Aiki (Wasa/Simintin): Wannan hanya ce ta kusa don yin haske mai inganci na ainihin lokaci. Ba da fifikon haɗa ƙa'idodin NieR cikin tsarin 3DGS don samfuran kadarori ko yanayin simintin inda daidaiton haske yana da mahimmanci ga aminci (misali, simintin firikwensin).
3. Ga Masu Zuba Jari: Aikin yana nuna cikar 3D Gaussian Splatting daga sabon kayan aikin gani zuwa ingantaccen injin simintin ƙwararru. Kamfanonin da ke gina simintin tuƙi mai sarrafa kansa (misali, NVIDIA DRIVE Sim, kayan aikin simintin Waymo) yakamata su sanya ido sosai akan wannan zuriyar.

Bincike na Asali (300-600 kalmomi): Tsarin NieR yana wakiltar muhimmin mataki a cikin rufe gibin tsakanin saurin gudu na 3D Gaussian Splatting (3DGS) da ƙaƙƙarfan buƙatun yin haske na zahiri (PBR). Kamar yadda aka lura a cikin aikin farko akan wakilcin yanayi na jijiyoyi ta Mildenhall et al. (NeRF), babban ƙalubale shine daidaita ingancin lissafi tare da ikon ƙirar tasirin dogaro da kallo mai rikitarwa. 3DGS na gargajiya, ga duk fa'idodinsa, sau da yawa ya gaza a nan, yana ɗaukar hulɗar haske a matsayin matsala ta matsakaicin ƙididdiga. Gabatarwar NieR na sashen rarraba haske bisa ga yanayin yanayi martani ne kai tsaye ga wannan iyaka. Yana haɗa ƙirar inuwa da ke tunawa da waɗanda aka yi amfani da su a cikin masu yin haske kamar RenderMan ko injuna na ainihin lokaci kamar tsarin kayan aikin Unreal Engine, amma a cikin tsarin 3DGS na tushen ma'ana, na tushen ma'ana. Wannan ba kawai ingantaccen ado ba ne; kamar yadda bincike daga cibiyoyi kamar MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) ya jaddada, daidaitaccen simintin haske yana da mahimmanci don horarwa da tabbatar da tsarin hangen nesa na kwamfuta, musamman a cikin yankuna masu mahimmanci ga aminci kamar motoci masu sarrafa kansu. Haske mai shuɗewa ko kuskure a kan mota na iya yaudarar ƙirar hangen nesa na kiyasin nisa ko nau'in kayan aiki. Sashen Haɓaka Matsakaicin Yanayin Yanayin a Matakai (HNGD) yana da fahimta daidai. Ya wuce haɓaka da ya dogara da kallo da aka saba yi a cikin 3DGS, wanda zai iya zama maras kwanciyar hankali a ƙarƙashin haske mai motsi. Ta hanyar haɗa haɓaka zuwa rikitarwar lissafi na ciki (bambancin yanayin yanayi), NieR ya gina wakilcin yanayi mai ƙarfi da kuma gabaɗawa. Wannan ya yi daidai da yanayin a cikin fanni mai faɗi, kamar yadda aka gani a cikin ayyuka kamar Mip-NeRF 360, waɗanda kuma ke amfani da siginonin lissafi don jagorantar ingancin wakilci. Duk da haka, hanyar tana da iyakoki. Dogaro da tsarin yanayin saman, wanda dole ne a ƙiyasta ko a ba da shi, yana gabatar da yuwuwar tushen kuskure. Bugu da ƙari, yayin da yake ƙware a hasken haske kai tsaye, ƙirar don watsawa $L_d$ ya kasance mai sauƙi, yana iya yin watsi da ƙananan abubuwa na hasken kai tsaye da kuma toshewar yanayi waɗanda ke da mahimmanci don cikakken hoto na gaske. Idan aka kwatanta da ayyukan da ke tare da binciken filayen haske a cikin wakilcin Gaussian, NieR ya zaɓi ingantaccen haɗawa, sarrafa haɗin ƙa'idodin zane-zane, yana sa gudummawar sa da iyakoki su bayyana a fili. A zahiri, NieR baya neman sake ƙirƙira ma'aunin yin haske amma don saka mafi tasiri na sassansa—haske mai haske wanda tsarin yanayin yanayi ke motsa shi—cikin tsarin yin haske mafi sauri da ake samu a yau. Wannan injiniyanci mai aiki yana sa ya zama gudummawar da ke da sha'awa sosai tare da yuwuwar aikace-aikacen nan take.

7. Aikace-aikace na Gaba & Hanyoyin Bincike

Aikace-aikace na Nan Take:

• Simintin Tuƙi Mai Ingantaccen Inganci: Don horarwa da gwada tarin fahimtar ADAS/AV, inda daidaitaccen yin haske na wasu motoci (mai haske), tituna masu jika, da alamomin zirga-zirga suke da mahimmanci.
• Nunin Samfura & Kasuwancin E-commerce: Yin haske na ainihin lokaci, na gaske na kayan masarufi tare da kayan aiki masu rikitarwa kamar na'urorin lantarki masu goge-goge, kayan ado, ko fentin mota.
• Samarwa na Virtual: Sauri, yin haske na yanayi na gaba da yuwuwar yin haske na baya a rayuwa inda hulɗar haske tare da kayan aiki yana buƙatar zama mai motsi da aminci.

Hanyoyin Bincike:

1. Haɗawa tare da Cikakken Haske na Duniya: Tsawaita sashen LD don ƙirar hasken kai tsaye sau ɗaya ko haɗawa tare da dabarun ajiye haske.
2. Gyaran Kayan Aiki & Sake Yin Haske: Yin amfani da sifofin da aka rarraba $k_s$, $L_s$, $L_d$ don gyaran kayan aiki bayan ɗauka da sake yin haske na yanayi mai motsi.
3. Wakilcin Haɗin Kai don Kadarorin Jijiyoyi: Bincika idan Gaussian da aka ƙarfafa NieR zai iya zama tsarin kayan aiki na duniya wanda ke ɓoye duka lissafi da ƙirar kayan aiki na asali, ana iya amfani dashi a cikin injunan yin haske daban-daban.
4. Bayan Tsarin Gani: Yin amfani da ƙa'idar rarraba bisa ga yanayin yanayi ga sauran simintin firikwensin kamar dawowar ƙarfin LiDAR ko ƙirar giciye na radar, waɗanda kuma suke da tasiri sosai ta hanyar alkiblar saman da kayan aiki.

8. Nassoshi

1. Wang, H., Wang, Y., Liu, Y., Hu, F., Zhang, S., Wu, F., & Lin, F. (2024). NieR: Normal-Based Lighting Scene Rendering. arXiv preprint arXiv:2405.13097.
2. Kerbl, B., Kopanas, G., Leimkühler, T., & Drettakis, G. (2023). 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics, 42(4).
3. Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020). NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV.
4. Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021). Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. ICCV.
5. Kajiya, J. T. (1986). The Rendering Equation. ACM SIGGRAPH Computer Graphics, 20(4).
6. Zhu, J., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. ICCV.
7. NVIDIA. (2023). NVIDIA DRIVE Sim. Retrieved from https://www.nvidia.com/en-us/self-driving-cars/simulation/