Advancing Solar Energy: Deep Learning for Precise Irradiance Nowcasting Using Sky Images

      Solar photovoltaics (PV) have become a cornerstone of the global renewable energy landscape, significantly increasing their share in power generation over the past five years. This rapid expansion, however, brings a heightened demand for sophisticated solutions to predict and manage the inherent variability in solar energy production. Accurate forecasting of solar irradiance, particularly for short-term horizons, is vital for maintaining grid stability, optimizing energy trading, and efficiently managing energy storage systems like batteries or hydrogen facilities. Without precise predictions, operators face increased degradation of equipment, imbalance settlement costs, and operational penalties.

      A recent academic paper, "Deep Learning Multi-Horizon Irradiance Nowcasting: A Comparative Evaluation of Three Methods for Leveraging Sky Images" by Eriksen et al. (2026), investigates advanced deep learning (DL) techniques to tackle this challenge. The research focuses on "nowcasting," which involves predicting solar irradiance 0 to 30 minutes into the future, a critical window for real-time grid control and intraday energy market adjustments. The study critically evaluates how different methods of integrating data from All-Sky Imagers (ASIs)—ground-based cameras that capture comprehensive views of the sky—can enhance the accuracy of these short-term forecasts.

The Crucial Role of Irradiance Nowcasting

      Irradiance nowcasting is the art and science of predicting the immediate future of solar radiation. Unlike longer-term weather forecasts, nowcasting deals with the rapidly changing dynamics of cloud cover, which can cause significant and sudden fluctuations in solar power output. For operators of solar power plants and grid managers, even a few minutes of accurate foresight can translate into substantial operational advantages and financial savings.

      Such forecasts are instrumental in several key areas. For power plant and grid control, they enable proactive adjustments to maintain a balanced supply and demand. In energy markets, better predictions facilitate more precise trading, minimizing the risk of penalties associated with forecast errors and optimizing imbalance settlements. Furthermore, for integrated systems combining PV with energy storage, precise nowcasting helps manage charging and discharging cycles, preventing the unregulated power generation variability that can lead to accelerated system degradation. The ability to forecast accurately in this short-term window is paramount for maximizing the benefits and mitigating the risks associated with large-scale solar energy deployment.

Leveraging All-Sky Imagers for Enhanced Prediction

      All-Sky Imagers (ASIs) are pivotal tools in improving nowcasting accuracy. These specialized cameras continuously capture images of the entire sky, providing real-time visual data on cloud presence, type, and movement. By analyzing these images, AI systems can anticipate when clouds will obscure the sun, thus predicting dips in solar irradiance before they occur. This visual data complements traditional irradiance measurements, offering a rich source of information about atmospheric conditions directly influencing PV performance.

      The research paper highlights how modern approaches to DL nowcasting heavily rely on ASIs, often combining their visual input with historical irradiance time-series data. Convolutional Neural Networks (CNNs) are frequently employed due to their inherent ability to process and extract meaningful spatial features from images, such as cloud patterns and textures. However, the exact methodology for integrating this visual data, and which specific image qualities yield the most useful information, remains an active area of investigation.

Comparative Evaluation of Deep Learning Methods

      The study undertaken by Eriksen et al. (2026) compared three distinct methods for incorporating ASI images into a deep learning model for solar forecasting. The underlying deep learning architecture for temporal forecasting was based on Long Short-Term Memory (LSTM) networks, known for their proficiency in handling sequential data. For image processing, a Convolutional Neural Network (CNN) module was often integrated to handle the spatial information from the ASI images. These hybrid CNN-LSTM architectures are a promising path for separately extracting valuable spatial and temporal patterns from diverse data sources.

      The three methods evaluated were:

• Method A: Raw RGB Image Processing. This approach utilized a CNN to directly extract features from unprocessed, raw RGB (red, green, blue) images captured by the ASIs. The CNN was tasked with identifying relevant visual cues from the raw pixel data, feeding these learned features into the LSTM model.
• Method B: Domain-Knowledge-Informed 2D Feature Maps. This method introduced an intermediate step of "feature engineering." State-of-the-art algorithms were employed to create 2D feature maps from the raw images. These maps encoded specific, domain-knowledge-driven information such as cloud segmentation (identifying cloud boundaries), cloud motion vectors (tracking cloud movement), solar position, and stereoscopic cloud base height (determining cloud altitude). These intelligently engineered 2D feature maps were then passed to a CNN to extract more refined, compound features before being fed into the LSTM.
• Method C: Aggregated Engineered Features as Time-Series Input. Building upon Method B, this final approach aggregated the engineered 2D feature maps into time-series data. Instead of feeding 2D maps directly into a CNN, key insights derived from these maps (e.g., average cloud cover, dominant cloud motion direction, average cloud height) were summarized into a sequential format. This aggregated time-series data was then directly input into the LSTM model.

      All three methods were trained on a high-frequency, 29-day dataset of global horizontal irradiance (GHI) and ASI images, generating multi-horizon forecasts up to 15 minutes ahead. Their performance was then rigorously evaluated using standard metrics like root mean squared error (RMSE) and skill score over seven selected days, encompassing various atmospheric conditions.

Key Findings and Business Implications

      The comparative evaluation revealed a significant insight: Method C, which leveraged aggregated engineered ASI features as time-series input, consistently delivered superior forecasting performance. This finding is crucial because it demonstrates that effective integration of ASI images into DL nowcasting models can be achieved without relying solely on complex, spatially-ordered DL architectures, at least for the specific nowcasting task. This highlights an opportunity for alternative image processing methods that may simplify model complexity while maintaining or even improving accuracy.

      For businesses and organizations relying on solar energy, these findings have profound implications:

• Optimized Resource Allocation: More accurate nowcasts enable better management of energy resources. Utilities can anticipate shortfalls or surpluses from solar generation, allowing for more efficient dispatch of conventional power plants or better integration with energy storage solutions.
• Reduced Operational Costs: Minimizing forecast errors directly reduces penalties in energy markets and avoids costly last-minute adjustments. This translates into tangible financial savings for operators of solar farms and energy traders.
• Enhanced Grid Stability: With better visibility into immediate solar output fluctuations, grid operators can proactively stabilize the power grid, preventing outages or brownouts and ensuring reliable energy supply.
• Scalability and Deployment: The success of aggregated features suggests that nowcasting solutions can be more flexible in their deployment. This could involve edge AI devices that perform initial image processing and feature aggregation locally, sending only the most relevant summarized data to a central forecasting system. This strategy aligns perfectly with solutions like ARSA's AI Box Series, which offers plug-and-play edge AI systems for rapid, on-site deployment and processing.

      The research also contributed to expanding the applicability of nowcasting methods northward by evaluating models at a high latitude (approximately 60° N), demonstrating the robustness of these techniques across diverse geographical locations and varying solar conditions.

The Future of AI in Energy Nowcasting

      The study underscores the continued importance of integrating domain knowledge into AI systems. While purely data-driven deep learning models are powerful, incorporating expert understanding through engineered features can often lead to more robust and accurate results. This balance between raw data processing and intelligent feature extraction is likely to drive future innovations in AI for renewable energy. The superior performance of aggregated engineered features also opens doors for exploring even more sophisticated spatial DL feature processing methods, pushing the boundaries of what is possible in nowcasting.

      For enterprises aiming to harness the full potential of AI and IoT in their operations, practical deployments of advanced analytics are crucial. Solutions like ARSA AI Video Analytics can be customized to process complex visual data, such as from ASIs, and integrate these insights into broader operational dashboards. Whether deploying on-premise for full data control and privacy, or leveraging edge systems for low-latency processing, the ability to transform passive data into active intelligence is invaluable.

      To learn more about implementing advanced AI and IoT solutions for enhanced operational intelligence and decision-making, we invite you to contact ARSA for a free consultation.

      **Source:** Eriksen, E. W., Nyg˚ard, M. M., Erdmann, N., & Riise, H. N. (2026). Deep Learning Multi-Horizon Irradiance Nowcasting: A Comparative Evaluation of Three Methods for Leveraging Sky Images. Retrieved from https://arxiv.org/abs/2603.26704