tag:blogger.com,1999:blog-5801421492525605252024-03-13T20:57:12.072+01:00kugelfishA TechnoLogBernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comBlogger127125tag:blogger.com,1999:blog-580142149252560525.post-1500037403989531402024-02-24T18:39:00.006+01:002024-03-08T22:49:16.486+01:00Long-distance Interconnects and their Contribution to Grid Balance<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNVRT9Ffwk6k0ebxq1ei7iiuYQqLHtMrTa2mkOJ3vQfoXs7VZ8tMNzgLZPshfOjmEyFnZAXrwVuO9hvTEYLD7vVsyuQUPtXY3q6QaLgRxFoAoZQUit5_vbPj39bAgM3F7oJ4PuiN5n4nyKQx4EWVTBQq8UCOPKTaxaMGhYNXvA1W1Y00NGXcgxcnlWjM47/s1024/laufenburg.jpg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="656" data-original-width="1024" height="294" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNVRT9Ffwk6k0ebxq1ei7iiuYQqLHtMrTa2mkOJ3vQfoXs7VZ8tMNzgLZPshfOjmEyFnZAXrwVuO9hvTEYLD7vVsyuQUPtXY3q6QaLgRxFoAoZQUit5_vbPj39bAgM3F7oJ4PuiN5n4nyKQx4EWVTBQq8UCOPKTaxaMGhYNXvA1W1Y00NGXcgxcnlWjM47/w459-h294/laufenburg.jpg" width="459" /></a></div>In <a href="https://blog.kugelfish.com/2024/01/balancing-renewables-grid-across.html" target="_blank">this previous post</a>, we looked at how much of the current electricity demand could be met directly if the entire production were based on solar and wind. The observation was that depending on the country, about 70-80% could be covered in real-time by scaling up an optimal mix based on the current solar & wind generation capacity. This would leave about 20-30% of unmatched load at some times and the same amount of surplus at other times.<p></p><p>In <a href="https://blog.kugelfish.com/2024/02/vehicle-to-grid-for-long-duration.html" target="_blank">this post</a>, we looked at how storage and in particular under-utilized car batteries could be used to help balance the grid between times of over and under-production.</p><p>Another potential approach of increasing the balance between supply and demand would be to improve geographic diversity across wind or solar generation beyond the <a href="https://en.wikipedia.org/wiki/Synoptic_scale_meteorology" target="_blank">synoptic scale</a> of common weather pattern which is in the order of a thousand or more kilometers.</p><p>The image above shows the site of the <a href="https://ethw.org/Milestones:Star_of_Laufenburg_Interconnection,_1958" target="_blank">Laufenburg substation</a>, where in 1958 the national grids of France, Germany and Switzerland were interconnected for the first time, setting the foundation for todays <a href="https://en.wikipedia.org/wiki/Synchronous_grid_of_Continental_Europe" target="_blank">continental European synchronous grid</a> spanning from Portugal to Ukraine and from Scandinavia to Turkey or Marocco.</p><p>While the synchronous interconnection of AC high voltage transmission grids greatly increases the efficiency and resilience for each of the participating transmission system operators most of the energy continues to flow mostly over short distances between neighboring zones.</p><p>Technological breakthroughs in high-voltage DC (<a href="https://en.wikipedia.org/wiki/High-voltage_direct_current" target="_blank">HVDC</a>) switching and conversion systems have brought HVDC interconnects from being an expensive niche solution to <a href="https://spectrum-ieee-org.cdn.ampproject.org/c/s/spectrum.ieee.org/amp/multiterminal-hvdc-networks-2666634970" target="_blank">a core technology of new high-capacity and long-distance super-grids emerging just about now</a>. Key advantages of HVDC transmission are the lower loss rates and the ability to run a very high voltages across coaxial underwater and underground cables, making them a suitable solution to transport large amounts of energy across long distances.</p><p>The recently inaugurated <a href="https://en.wikipedia.org/wiki/Viking_Link" target="_blank">Viking Link</a> connecting the UK and Denmark across the North Sea is currently the longest operating next-gen HVDC interconnect in Europe at 765km carrying up to 1.4GW of power in either direction. Germany started construction on the <a href="https://de.wikipedia.org/wiki/Suedlink">SuedLink</a> project with several 4GW underground HVDC links connecting the German North-Sea coast with the industrial and population centers further south. Maybe the most ambitious among the many HVDC projects across Europe is the 3800km 3.8GW <a href="https://en.wikipedia.org/wiki/Xlinks_Morocco%E2%80%93UK_Power_Project" target="_blank">Xlinks</a> project connecting solar and wind farms in Marocco to consumers in the UK via a submarine HVDC cable. A similar building boom of long-distances HVDC links can be observed for the last few decades <a href="https://en.wikipedia.org/wiki/List_of_HVDC_projects">around the world</a>.</p><p>In order to see how much interconnection beyond the synoptic scale of weather systems could automatically contribute to the balancing of supply and demand at either end, we are using a joint LP optimization problem across sets of 2 countries similar to the one used for <a href="https://blog.kugelfish.com/2024/01/balancing-renewables-grid-across.html" target="_blank">this post</a>.</p><p>First we are looking at an interconnection of Germany and Spain. Both among the largest energy markets in Europe with significant geographic dimensions and they are over a thousand kilometers apart from each other and are at least partially in different climate zones. By varying the available interconnection capacity from zero to 32 GW, we can see the import/export flows increase and the unmatched shortfalls decrease accordingly:</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhECIGp0RtSyPwcQulzO9f9hx7FO04LLVuxbwXZnGm_KO3Dj70v19idjZISJf2bRISCUhcDnTqTUaOZ3GUmuMZbCr-m4x3-XahFjshId1A8u5VonsGZNNBVQZd56W-OYTSK_5J1z4RJO9MCLv7v2Ek7YNeg3xLW54pdutFbTYpMzG6caEVOEn4Vjibn_2_B/s604/de_es_link.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="604" data-original-width="603" height="467" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhECIGp0RtSyPwcQulzO9f9hx7FO04LLVuxbwXZnGm_KO3Dj70v19idjZISJf2bRISCUhcDnTqTUaOZ3GUmuMZbCr-m4x3-XahFjshId1A8u5VonsGZNNBVQZd56W-OYTSK_5J1z4RJO9MCLv7v2Ek7YNeg3xLW54pdutFbTYpMzG6caEVOEn4Vjibn_2_B/w466-h467/de_es_link.png" width="466" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Germany-Spain Interconnection</td></tr></tbody></table><p>Looking at the shortfall across different timescales of Germany for example, we can see that most of the improvements wold be over relatively short time-scales - hours to weeks.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid_MC0yGYfBc7VbfVR2zsW0o5iOwV5aAawOUUyQhoNFdtB2UK-57WhlBM4EPdQa7J-VKF8uHoiFcUSZI1iLATzq3osU5OzkrM8M9dFg2ymwz6voBRmpB_OQOnqp4LlLRl5trG3ganYCq1ojoBFjVRBggGWv-BIogIlp1eaAi56l_elroB4tnGw4qGVcsD-/s623/de.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="591" data-original-width="623" height="457" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid_MC0yGYfBc7VbfVR2zsW0o5iOwV5aAawOUUyQhoNFdtB2UK-57WhlBM4EPdQa7J-VKF8uHoiFcUSZI1iLATzq3osU5OzkrM8M9dFg2ymwz6voBRmpB_OQOnqp4LlLRl5trG3ganYCq1ojoBFjVRBggGWv-BIogIlp1eaAi56l_elroB4tnGw4qGVcsD-/w481-h457/de.png" width="481" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Germany - imbalance across time-scales</td></tr></tbody></table><p>Both countries being still essentially being in a similar climate zone and having similar seasonality, does not seem to provide enough complementarity to significantly improve the imbalance across the very long seasonal time scales (month to quarters).</p><p>We can also try a more extreme example, connecting the Netherlands with Greece. Both are geographically smaller and distinctively different - with Netherlands on the shore of the North Sea and Greece in the South-Eastern Mediterranean. Both are also much smaller energy markets, which means that a much smaller interconnection capacity would be needed take advantage of the interchange of matching surplus and shortfalls.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpptRRH9mKWtnFBzlXDkpKUy162IuhUMGSH2mW-YXMmJaBau_f-BRU6p5BLQ-Klg_rSfMhEjcIa1LEglxiGk31C0NFc7wGUbXxSAepz_b3m1FqDw3o2uVlTzfZnA_iTakn1QDxBGDf-pp_eTXYh3l4S4B4wzVZxXbCaUtOuVeoH9Nxxy8kvZMSqlxDzzVS/s612/nl_gr_link.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="598" data-original-width="612" height="464" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpptRRH9mKWtnFBzlXDkpKUy162IuhUMGSH2mW-YXMmJaBau_f-BRU6p5BLQ-Klg_rSfMhEjcIa1LEglxiGk31C0NFc7wGUbXxSAepz_b3m1FqDw3o2uVlTzfZnA_iTakn1QDxBGDf-pp_eTXYh3l4S4B4wzVZxXbCaUtOuVeoH9Nxxy8kvZMSqlxDzzVS/w475-h464/nl_gr_link.png" width="475" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Netherlands-Greece Interconnection</td></tr></tbody></table><p>Despite the more extreme scenario, we only see a slightly higher relative import/export flow and corresponding decrease in unmatched load across both markets.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWLylFVIQept961ApbpkMb69kUMUF7tYvNgAaeD1bMcir5SulP3ybMkMr_lhqCPVgtNPdkGNx6ufGvaXGnjXf4bE5B5VSW-x4Hccxagr4UPPhywARmA5GBOoKBNnsJWTV1WIjnHt5z1hclXX6k-El6r8bCNYiQW5GAQdDPMOUICzRhvbhA9B732GnqQ3gU/s600/nl.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="579" data-original-width="600" height="439" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWLylFVIQept961ApbpkMb69kUMUF7tYvNgAaeD1bMcir5SulP3ybMkMr_lhqCPVgtNPdkGNx6ufGvaXGnjXf4bE5B5VSW-x4Hccxagr4UPPhywARmA5GBOoKBNnsJWTV1WIjnHt5z1hclXX6k-El6r8bCNYiQW5GAQdDPMOUICzRhvbhA9B732GnqQ3gU/w454-h439/nl.png" width="454" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Netherland - imbalance across time-scales</td></tr></tbody></table><p>With this experiment we can show that introducing some additional geographic diversity in wind & solar production by naively interconnecting 2 power-grids over distances of more than thousand kilometers could reduce the load & production imbalance by 5-10%. A less arbitrary choice of interconnection points would probably increase the potential impact.</p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-34865395092354361292024-02-07T00:40:00.004+01:002024-02-08T13:17:03.892+01:00Vehicle-to-Grid for Long Duration Storage?<p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFcH8t5PUWPN65flYQPLDn2qOt-1Zb3x_1IYhqXa24vxDFfRrTI6JE9DcjGu5i5F6noZFkUL5bX0dLNFAZKxlf1ShfVG2CUuce58wXMmo75kcqwUTbpaVXdpZwkvb8I4T25kt-0WXvpBMukkfLpCOWIbQ_ek_tCOohfykSyY_Mzezg7nAFYXqTseqYZvn0/s2192/IMG_20200423_152000.jpg" style="clear: left; display: inline; float: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="2192" data-original-width="2048" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFcH8t5PUWPN65flYQPLDn2qOt-1Zb3x_1IYhqXa24vxDFfRrTI6JE9DcjGu5i5F6noZFkUL5bX0dLNFAZKxlf1ShfVG2CUuce58wXMmo75kcqwUTbpaVXdpZwkvb8I4T25kt-0WXvpBMukkfLpCOWIbQ_ek_tCOohfykSyY_Mzezg7nAFYXqTseqYZvn0/w362-h388/IMG_20200423_152000.jpg" width="362" /></a></p>Most cars stand around most of the time.<div><br /></div><div>The <a href="https://blog.kugelfish.com/2023/05/it-is-12-noon-do-you-know-where-you-car.html" target="_blank">average driving distance per car is 30-40 km </a>- even in the US and trips over 100km are quite rare (<a href="https://transportgeography.org/contents/chapter5/road-transportation/distribution-car-trips-distance-united-states/" target="_blank">less than 4%</a>). Assuming, some number of heavily used vehicles - taxis, service vehicles etc. means that there is a long tail of vehicles which might not move at all for several days except maybe the occasional trip on a week-end.<p></p><p>Assuming some future state where (nearly) all cars are going to be electric, we will have a very large number of big batteries on wheels mostly standing around mostly under-utilized and already paid for by their owners.</p><p>As we have seen in <a href="https://blog.kugelfish.com/2024/01/balancing-renewables-grid-across.html" target="_blank">this previous post,</a> there would be a significant imbalance on a weekly timescale between load and generation in a primarily wind & PV based power grid. Given the large amount of already paid for storage capacity, it would be temping to use it as grid storage (V2G vehicle-to-grid) over these time-ranges where investment in dedicated grid storage might be hard to economically justify.</p><p>There is already a significant amount of literature estimating the potential impact of V2G for short-duration, intra-day grid storage. But the use cases could go well beyond that. Assuming that car owners could "donate" 25% of their car's battery capacity into a grid storage pool as long as it is not used, how long would it take before this reserve would really be needed for driving? A few days, maybe weeks or only once or twice a year for a vacation trip?</p><p>To see the potential benefits, we are adding a V2G component to the previous system <a href="https://blog.kugelfish.com/2023/07/linear-programming-with-energy-data.html" target="_blank">cost optimization LP model</a>. For sizing the V2G model, we are using the assumptions given in section 2.2 of this <a href="https://www.swiss-emobility.ch/de-wAssets/docs/ETH-STudie-vehicle-to-grid/Report_V2G_Study-3.pdf" target="_blank">research report</a>.</p><p>Using again the Netherlands as an example, there are about 10 million cars in service today and we assume all of them be electrical in some near future. We assume that at any time about 60% of vehicles are participating in the V2G storage pool and are currently plugged in - 80% of which support bi-directional charging. We assume the average battery size per car to be 100kWh and the capacity made available to the grid is only 25% of each participating car. The maximum charge/discarge power is limited to 2kW per car, which is a small fraction of a typical 11 or 22kW wall-box home charger</p><p>As we have seen for the same model in <a href="https://blog.kugelfish.com/2023/09/energy-storage-and-flexible-demand-two.html" target="_blank">this previous post</a>, demand side flexibility can significantly reduce or eliminate the need for fast short-term storage to balance the grid at hours to day time-scales. On top of this model, we now add the essentially free of cost V2G capacity.</p><p>
</p><p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;"></th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">baseline</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">40% flex demand (12h)</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">40% flex + v2g</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Load (total / avg / peak)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation (total / avg / peak)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">116.4TWh / 13.3GW / 48.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">114.2TWh / 13.1GW / 47.8GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">110.8TWh / 12.7GW / 46.2GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation PV / ONW / OFFW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.4% / 60.6% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">38.3% / 61.7% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">36.1% / 63.9% / 0.0%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Annual Cost / Cost per MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.5B€ / 85.3 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.9B€ / 79.8 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.7B€ / 77.4 €/MWh</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">System Efficiency</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">85.5%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">87.1%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">89.9%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Surplus</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.5%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.3%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0.7%</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Storage contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">28.2TWh (28.4%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">17.6TWh (17.7%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18.6TWh (18.7%)</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Power</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.6GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Capacity / Duration</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.9GWh / 5.2h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS contribution</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.5TWh (9.5%) 236 cycles</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Power (charge/ discharge)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.1GW / 11.0GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">13.5GW / 10.6GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11.1GW / 9.0GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Capacity / Duration</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5489.6GWh / 500h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5431.4GWh / 513h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5276.1GWh / 583h</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18.8TWh (18.9%) 3 cycles</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">17.6TWh (17.7%) 3 cycles</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">12.0TWh (12.0%) 2 cycles</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">V2G Power</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.6GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">V2G Capacity</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">120.0GWh (12h)</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">V2G contribution</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">6.6TWh (6.6%) 55 cycles</td></tr></tbody></table><p></p><p>While the available V2G storage pool is about 3 times the size of the short term battery storage needed for the inflexible baseline scenario, the impact on the capacity needs for the capital expensive and rarely used long & seasonal duration storage is barely reduced, even though the V2G pool takes up about one third of the remaining storage contribution at a fairly high round-trip efficiency (80-90% vs. less than 60%).</p><p>Another perspective would be the reduction in total annual system cost (8.5B€ for baseline configuration). By allowing 40% of demand across all sector to shift by up to 12 hours in a most globally optimal way, we could reduce annual system cost by 600M€ or 7%. By adding V2G on top of that we arrive at a saving of 900M€ or 10.5%. This gain in global welfare can be distributed any which way among stakeholders like electricity consumers, the government, power grid operators or as an incentive for those who are supposed to contribute a "free" resource (flexibility, spare battery capacity) for the greater good of the community.</p><p>
</p><p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;"></th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">baseline</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">v2g</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Load (total / avg / peak)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation (total / avg / peak)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">116.4TWh / 13.3GW / 48.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">114.4TWh / 13.1GW / 47.9GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation PV / ONW / OFFW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.4% / 60.6% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">38.3% / 61.7% / 0.0%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Annual Cost / Cost per MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.5B€ / 85.3 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.9B€ / 79.3 €/MWh</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">System Efficiency</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">85.5%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">87.0%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Surplus</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.5%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0.9%</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Storage contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">28.2TWh (28.4%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">28.2TWh (28.4%)</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Power</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.6GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Capacity / Duration</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.9GWh / 5.2h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS contribution</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.5TWh (9.5%) 236 cycles</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Power (charge/ discharge)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.1GW / 11.0GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">12.6GW / 9.8GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Capacity / Duration</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5489.6GWh / 500h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5498.8GWh / 560h</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18.8TWh (18.9%) 3 cycles</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.3TWh (14.3%) 2 cycles</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">V2G Power</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.6GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">V2G Capacity</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">120.0GWh (12h)</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">V2G contribution</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">--</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.0TWh (14.0%) 116 cycles</td>
</tr>
</tbody>
</table><p></p>
<p>If we were to implement a full V2G pool without also taking advantage of demand flexibility first, we get a similar benefit as for demand flexibility alone, also increasing the number of average storage cycles for the V2G cluster from 55 to 116. Limiting the number of storage cycles and the charge/discharge power intensity can also help limit battery wear to not unnecessarily shorten the lifespan of the battery below the natural service lifetime of the car.</p><p>Given that demand flexibility is likely much easier to implement and operate than V2G, the conclusion from this analysis would be that policy makers and grid operators should focus on flexibility first. V2G can be a nice optional ad-on - generating about 50€ of annual value per car. Not enough to justify any significant investment, unless bi-directional charging is becoming the standard anyway because car owners enjoy the other benefits: using the car as a mobile power supply (V2L) or as an emergency <a href="https://www.generac.com/for-homeowners/home-backup-power" target="_blank">backup power source for their house</a> or to optimize the behind the meter utilization of rooftop solar power instead of selling it back at feed-in tarifs which can be much lower than retail power tarifs.</p></div>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-28928828335217846622024-01-29T00:17:00.004+01:002024-02-05T18:33:18.176+01:00Balancing a Renewables Grid across multiple Timescales<p>For a hypothetical future power grid where production is dominated by variable renewable sources like PV and wind, the challenge of balancing supply and demand at all times is well know. Using again the <a href="https://blog.kugelfish.com/2023/07/linear-programming-with-energy-data.html" target="_blank">ENTSO-E open-source data and Python linear programming solver</a>, we extrapolate the grid level solar and wind forecast data to cover the entire yearly electricity demand while minimizing the sum of the absolute errors between supply and demand for all hours in a year.</p><p>In order to turn the problem of minimizing the sum of absolute values into a linear programming problem, we use the bounding technique presented in <a href="https://yetanothermathprogrammingconsultant.blogspot.com/2017/11/lp-and-lad-regression.html" target="_blank">this tutorial</a>.</p><p>Using for example the data for the German power grid from Spring 2022 to Spring 2023, the optimal generation mix according to the above criteria is about 75% wind and 25% PV (blue line). With this configuration about 80% of the annual supply & demand of 475 TWh would be matched at the hourly level, leaving a discrepancy of 20% or 107TWh to be equalized somehow.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOMEbGllYWfJ9rnoGPpRFpv60sCUQFMalye2FyG7r91pyojylmr3tphCqgPtMAci9B7VrQbDu33rezVRFcqFVjod-Y21_ObmtkPrkXOCG1WAW0EOPPFAePdGY7BXVOsn4aBUE4NEae0TyRYXsgxfO_9d99eqYZc_nHA8_M8EFwa4pLL2cJUQ8TLqRPBx0w/s619/storage_horizon.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="596" data-original-width="619" height="503" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOMEbGllYWfJ9rnoGPpRFpv60sCUQFMalye2FyG7r91pyojylmr3tphCqgPtMAci9B7VrQbDu33rezVRFcqFVjod-Y21_ObmtkPrkXOCG1WAW0EOPPFAePdGY7BXVOsn4aBUE4NEae0TyRYXsgxfO_9d99eqYZc_nHA8_M8EFwa4pLL2cJUQ8TLqRPBx0w/w522-h503/storage_horizon.png" width="522" /></a></div><p>For different time-scales, the diagram shows how much imbalance is left after if by some way we could equalize the imbalance up to some time-scale - for example 4 or 12 hours, days, weeks months etc. We can see for example that for the optimal mix, the greatest drop in unmatched imbalance is in between the order of days to weeks, which matches the cycles of the general weather situation in Europe which includes periods with more or less wind and more or less clouds and precipitations. The residual imbalance beyond the month timescale is less than 5% or 20TWh, which means that there is hardly a seasonal imbalance.</p><p>Compared to the optimal mix, a 100% PV generation would be much more imbalanced: 30% intra-day plus another 30% seasonal, which require equalization across more than a 3 months. A 100% Wind (optimal onshore & offshore mix) is much less volatile across hours of the day and generally follows the seasonality of demand (more demand & generation in winter).</p><p>The optimal mix and resulting imbalance is surprisingly similar across various European countries with different climatic conditions: from the north-sea cost of Netherland to the southern mediterranean Greece to to the semi-arid plains of Spain.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWD1b417Vqmt-n0hNS_nOwox0UenZio81DKdg2kl518cVUC7KlAw5nI4bPKrmYpYAdpHzs7CJZg60RXxqUvpdboM_WwM0kj8ZhUm4EUSDZBcv_4UcxLYpyQdk4u7iruIdaawuTUcI4TnKYWJVFe0y_9ZooGZyfQLXxRIWNJX38g01OrxJ6TqyT_adadCCq/s613/storage_horizon_country.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="596" data-original-width="613" height="497" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWD1b417Vqmt-n0hNS_nOwox0UenZio81DKdg2kl518cVUC7KlAw5nI4bPKrmYpYAdpHzs7CJZg60RXxqUvpdboM_WwM0kj8ZhUm4EUSDZBcv_4UcxLYpyQdk4u7iruIdaawuTUcI4TnKYWJVFe0y_9ZooGZyfQLXxRIWNJX38g01OrxJ6TqyT_adadCCq/w512-h497/storage_horizon_country.png" width="512" /></a></div><br /><div class="separator" style="clear: both; text-align: justify;"><span style="text-align: left;">The good news even without doing anything special an optimized combination of PV & wind can have a an effective load carrying capacity of around 80%, the bad news is that closing the remaining 20% will be much harder and much more expensive.</span></div><div class="separator" style="clear: both; text-align: justify;"><span style="text-align: left;"><br /></span></div><div class="separator" style="clear: both; text-align: justify;"><span style="text-align: left;">The available choices to correct the imbalance are either</span></div><div class="separator" style="clear: both; text-align: justify;"><ul><li style="text-align: left;">Over-capacity with overproduction and curtailment</li><li style="text-align: left;">Demand flexibility (shifting demand in time to match available supply)</li><li style="text-align: left;">Storage</li><li style="text-align: left;">Long-distance HVDC interconnects between regions with different weather pattern.</li><li style="text-align: left;">Other intermittent energy sources which have low with wind or solar.</li><li style="text-align: left;">Dispatchable renewable energy sources, e.g. hydro-electric.</li></ul><div style="text-align: left;">Demand flexibility can be a very effective option to match imbalances across a few hours or days, in the case of large-scale industrial heat storage maybe days to weeks. Storage is increasingly cost competitive in the short duration of hours to days. 4 hour duration grid scale battery storage plants are currently being constructed all over the world in significant numbers to provide lucrative grid management services (frequency response, synthetic inertia, black start capabilities etc.) as well as starting to balance intraday supply and demand in PV heavy grids (e.g. <a href="https://www.canarymedia.com/articles/energy-storage/a-huge-battery-has-replaced-hawaiis-last-coal-plant" target="_blank">Hawaii</a>). At this point, the cost per kWh of storage capacity is roughly at par or lower with closed-loop pumped hydro, the previously only viable large scale option for true electricity storage.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The economics of storage are become very quickly much more challenging with increasing timescales: any given unit of storage capacity can be amortized more quickly the more it can be cycled - from daily (365 times per year) to weekly (52 times per year), monthly (12 times per year) or even seasonly with only one cycle per year. To be profitable, seasonal storage capacity would have to be 350 times cheaper than intra-day storage capacity. The cheapest long-term storage a hydrogen storage in underground caverns ($2-$3 per KWh of capacity) or as liquid storage tanks for green Methanol, not including the high cost and low efficiency of the conversion equipment.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">For the longer time-scales - months to quarters, overbuilding generation capacity might end up being more cost effective. The chart below shows the same optimization for the same load and PV/wind potentials for the German grid assuming an annual overproduction between 0 to 20%:</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivunbItnbXQgyn58fYPvsci_iScVkkksborLMXXXtPdIIz29u01qn2a9nA6V_aaqdMV_eQIjYO45hpG_AvzkA0lxIL0vKKzJqCnHbyC6FWlzoV5rYLaNdJxK2X-7Vg8u1kSR9qP6cF9wmagqJhIsRck-sWA1dwV1xncY6j45uJg9yCx-HaO3N1LcB1wuVk/s627/storage_horizon_overcapacity.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="582" data-original-width="627" height="502" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivunbItnbXQgyn58fYPvsci_iScVkkksborLMXXXtPdIIz29u01qn2a9nA6V_aaqdMV_eQIjYO45hpG_AvzkA0lxIL0vKKzJqCnHbyC6FWlzoV5rYLaNdJxK2X-7Vg8u1kSR9qP6cF9wmagqJhIsRck-sWA1dwV1xncY6j45uJg9yCx-HaO3N1LcB1wuVk/w541-h502/storage_horizon_overcapacity.png" width="541" /></a></div><div style="text-align: left;">Around 5-10% overcapacity there is no more unmatched imbalance beyond the quarterly (3 month) horizon.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">For a <a href="https://blog.kugelfish.com/2023/09/energy-storage-and-flexible-demand-two.html" target="_blank">previous analysis</a>, we have optimized for minimal system cost using PV & wind generation with current LCOE production cost estimates 12-24h demand flexibility (for free) plus a storage cost estimate modelled after a grid scale battery storage plant or a (at this point hypothetical) H2 storage plant consisting of electrolyzers, underground cavern storage and combined-cycle gas turbines.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The resulting optimal mixes are closes to 40% PV and 60% wind (due to lower LCOE for utility scale PV) and a 20% over-production, mostly to compensate for storage losses.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">At the <a href="https://ember-climate.org/data/data-tools/carbon-price-viewer/" target="_blank">current cost of carbon emissions</a>, no existing storage technology can be cost competitive with traditional fossil fuel <a href="https://en.wikipedia.org/wiki/Peaking_power_plant" target="_blank">peaker plants</a> for compensating imbalanced beyond the timescale of days. But the charts above also show how increasing deployment renewable generation and short-term battery storage are squeezing out traditional peaker plants out of the lucrative segments into being a provider of capacity of last resort. <a href="https://www.swissolar.ch/de/news/detail/dreamteam-wasserkraft-und-solarstrom-44612" target="_blank">For special cases like Switzerland</a> flexibly dispatchable hydro-power plants with large dams and season storage reservoirs could be used to provide sufficient on demand capacity to balance demand and supply across a full yearly range of time-scales.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div></div>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-3224624231194469492024-01-08T22:51:00.004+01:002024-01-09T15:17:13.239+01:00Electrify all the Things!<p>While there is no technological silver-bullet to combat climate change, electrification is about as good as it gets - maybe at least a silver spoon to help keep the monsters at bay...</p><p>Today <a href="https://data.worldbank.org/indicator/EG.USE.COMM.FO.ZS?end=2015&start=2015" rel="nofollow" target="_blank">about 80% </a>of the world-wide energy consumption is still based on fossil fuels. The good news is that as part of a net zero transformation of the energy system, we do not need to replace all the fossil fuel inputs, but only their useful outputs.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiL36jxAlqqjlZVHCXvB2SQU3QObBggj0q7S8Df3kFEYx0FuSlzGiqWWrFDiaqHj9XBzNNrRx3uwtNfPRJkx4_IYQ-rrliMQClT0OHKbwpRqNYWFX3u72oGgvRGuo_hmLOqqYM_LoxBAon440Rch0ko_i47LHaEcYVCDefmGh9e075LZNEA3KjYYnhIqfpj/s3249/US%20Energy%202022.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1887" data-original-width="3249" height="372" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiL36jxAlqqjlZVHCXvB2SQU3QObBggj0q7S8Df3kFEYx0FuSlzGiqWWrFDiaqHj9XBzNNrRx3uwtNfPRJkx4_IYQ-rrliMQClT0OHKbwpRqNYWFX3u72oGgvRGuo_hmLOqqYM_LoxBAon440Rch0ko_i47LHaEcYVCDefmGh9e075LZNEA3KjYYnhIqfpj/w640-h372/US%20Energy%202022.png" width="640" /></a></div><br /><p>According to this Sankey energy flow diagram for the US in 2022 (source: <a href="https://flowcharts.llnl.gov/commodities/energy" rel="nofollow" target="_blank">LLNL</a>), about 20% of the system input is based on low CO2e energy sources. However this would correspond to 65% of the actually useful energy services provided by the system. Some of the most significant sources of loss are from the use of petroleum in the transport sector or from the electricity generation using natural gas or coal. </p><p>While generating more electrical power from modern renewable sources like wind and solar reduces the need for gas & coal by at least 2 times, assuming the average conversion efficiency of less than 50% across the fleet of existing fossil fuel plants. (The <a href="https://www.powermag.com/ge-ha-turbine-snags-another-world-record-for-ccgt-efficiency/" rel="nofollow" target="_blank">world record efficiency</a> for a modern combined-cycle natural gas burning plant is currently at about 63%).</p><p>However the biggest gains in efficiency comes from the direct electrification of all the processes which deliver useful energy services to their end users - typically in the form of movement or heat.</p><p>The largest aggregate fossil fuel consumer is the transportation sector, where the <a href="https://www.nytimes.com/interactive/2023/04/14/climate/electric-car-heater-everything.html" rel="nofollow" target="_blank">largest part</a> goes to moving cars and trucks.</p><p>While the typical pump-to-wheel efficiency of a gasoline powered vehicle is around 15-30% or 30-40% for diesel, the typical plug-to-wheel efficiency of a battery-electric vehicle is 50-80%. Taking the transportation sector average efficiency of 20% from the diagram above, electrifying road traffic would reduce its input energy needs by 2 to 5 times.</p><p>While burning natural gas, oil or coal for heat generation in households or industry has a yield of close to 100%, electrically powered heat pumps can reach effective yields of 200-500% by transferring ambient heat rather than conversion of input energy into heat.</p><p>The fossil fuel industry itself is also a very large energy consumer. The <a href="https://greet.anl.gov/publication-wft2tv3v" rel="nofollow" target="_blank">estimated well-to-tank efficiency</a> for Gasoline is about 80%, which means that 20% of the energy is used for extraction, transport and refining. Since gasoline production is a legitimate industrial activity, its energy use would be counted as energy service and not energy loss due to inefficiency.</p><p>Assuming an resolute push for large-scale direct electrification, the amount of low-carbon electricity generation would only about have to double from today in order to mostly eliminate fossil fuel use from the system globally.</p><p>At the other end of the spectrum, if we wanted to leave the energy consumers unchanged and replace all the current fossil fuels use with synthetic equivalents created from green hydrogen and captured C02 at less than 50% conversion efficiency, then we would need to add more than 10 times the amount of existing low-carbon electricity production.</p><p>By moving away from the stone-age technology of burning stuff, electrification has the potential to gain massive efficiencies and cut the global energy consumption in half.</p><p>While consistent electrification wherever possible is clearly the most credible path to decarbonize the global energy system, doing so is neither going to be cheap nor easy! However the consequences of not doing it would most likely be much more costly.</p><p><br /></p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-49923080286203221172023-09-02T11:00:00.048+02:002023-09-14T14:04:43.866+02:00Energy Storage and Flexible Demand - Two Sides of the Same Coin<p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjYcJ6d3QBjLy7SUViGeKKlbZNqxggNA7HfKWdVGqqKYbXRwyhRc3WkbwWLXW_f_8eKyaqKIqjF2EesQqCet7BpODjJyPPRkdegBW2vjuw509h5KAEaMKtGmW5P3HasKuGDSOVIC2VAE5ASDbgYoRzVUSR369nXPtBu-dQD-vLIJHLu3tBEKXhv1QPPIN1/s1370/PXL_20230807_193146092.jpg" style="clear: left; display: inline; float: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1370" data-original-width="1024" height="468" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjYcJ6d3QBjLy7SUViGeKKlbZNqxggNA7HfKWdVGqqKYbXRwyhRc3WkbwWLXW_f_8eKyaqKIqjF2EesQqCet7BpODjJyPPRkdegBW2vjuw509h5KAEaMKtGmW5P3HasKuGDSOVIC2VAE5ASDbgYoRzVUSR369nXPtBu-dQD-vLIJHLu3tBEKXhv1QPPIN1/w349-h468/PXL_20230807_193146092.jpg" width="349" /></a> For this <a href="https://blog.kugelfish.com/2023/07/solar-wind-storage-optimal-resource.html" target="_blank">previous post,</a> we have built a <a href="https://blog.kugelfish.com/2023/07/linear-programming-with-energy-data.html" target="_blank">very simplistic LP model</a> to determine the optimal resource allocation for a power grid that is based on wind, solar and storage.</p><p>This model has shown an obvious, significant need for storage to balance variable and non-dispatchable wind and solar generation to a fixed demand on timescales ranging from hourly to seasonal.</p><p>The key to decarbonizing high emission sectors like transport, buildings and industry is going to be aggressive electrification of these sectors, which is going to increase electricity demand significantly and put a strain on aging power distribution infrastructure.</p><p>The good news is that the vast majority of this new demand is going to be somewhat <a href="https://blog.kugelfish.com/2022/11/the-case-for-batch-mode-electricity.html" target="_blank">elastic and flexible in time</a> - potentially at no extract cost and without loss in comfort or quality of service.</p><p>Instead of using storage to match variable supply to a rigid load, we can exploit demand flexibility to match adaptive loads to existing supply and hence reduce the need to invest in expensive storage.</p><p>Most of the flexibility that comes at little or no additional cost is in fact taking advantage of already existing equivalent storage in another domain:</p><p></p><ul style="text-align: left;"><li>For transportation, the additional electricity will be used to charge batteries in electric vehicles, which are <a href="https://blog.kugelfish.com/2023/05/it-is-12-noon-do-you-know-where-you-car.html" target="_blank">typically quite oversized for the average daily use</a> and may only need to charged once a week - resulting in flexibility to shift the demand around in the order of days.</li><li>For building and industrial heat production, the additional electricity will be used to power heat pumps. Buildings which are heated or cooled can have significant inertia - thermal storage in the form of the buildings material and structure. Or there might still be a hot water storage tanks in the boiler room from previous systems or such thermal storage tanks could be added at much less cost than comparable electricity storage. </li><li>For industrial processes which need heat at higher temperatures, <a href="https://en.wikipedia.org/wiki/Thermal_energy_storage" target="_blank">specialised thermal storage systems</a> based on sand, volcanic rock or molten salt can also provide cost efficient heat storage at temperatures ranging from ambient to several hundred degree Celsius with a capacity to store heat for days or months.</li><li>For some energy intensive production processes, it might be possible to shift production to when electricity is most cheaply available and effectively store the energy in the warehouse as produced goods.</li></ul><p></p><p>In order to evaluate the potential impact of flexible demand, we extend the <a href="https://blog.kugelfish.com/2023/07/linear-programming-with-energy-data.html">previous model</a> to assume that a fraction of the existing load could be somewhat flexible in time. We are again using the actual load and wind/solar production data from the <a href="https://blog.kugelfish.com/2023/01/energy-data-analysis-with-pandas.html" target="_blank">ENTSO-E transparency platform</a> for the Netherlands between Spring 2022 to 2023 with a target availability of 100%.</p><p>We are assuming that some fraction of this load (0 to 60% respectively) which could be shifted in time anywhere from 0 to 12 hours. This would be a reasonably conservative assumption for both everyday electric car charging and many building heat applications:</p>
<p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;"></th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">20%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">40%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">60%</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Load (total / avg / peak)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation (total / avg / peak)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">116.4TWh / 13.3GW / 48.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">115.6TWh / 13.2GW / 48.5GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">114.2TWh / 13.1GW / 47.8GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">111.9TWh / 12.8GW / 46.8GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation PV / ONW / OFFW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.4% / 60.6% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.2% / 60.8% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">38.3% / 61.7% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">37.4% / 62.6% / 0.0%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Annual Cost / Cost per MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.5B€ / 85.3 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.2B€ / 82.3 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.9B€ / 79.8 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.8B€ / 78.1 €/MWh</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">System Efficiency</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">85.5%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">86.2%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">87.1%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">89.0%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Surplus</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.5%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.5%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.3%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0.8%</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Storage contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">28.2TWh (28.4%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">21.9TWh (22.0%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">17.6TWh (17.7%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">15.2TWh (15.3%)</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Power</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.6GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">2.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Capacity / Duration</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.9GWh / 5.2h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">16.4GWh / 5.7h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0GWh / 0.0h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0GWh / 0h</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS contribution</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.5TWh (9.5%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">3.4TWh (3.4%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0TWh (0%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0TWh (0%)</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Power (charge/ discharge)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.1GW / 11.0GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">13.9GW / 10.8GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">13.5GW / 10.6GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">12.8GW / 10.1GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Capacity / Duration</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5489.6GWh / 500h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5456.0GWh / 505h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5431.4GWh / 513h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5453.0GWh / 537h</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18.8TWh (18.9%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18.6TWh (18.7%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">17.6TWh (17.7%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">15.2TWh (15.3%)</td>
</tr>
</tbody>
</table><p></p>
<p>As one might expect, with a growing share of flexible load the need for short duration storage declines and goes away completely at about 50% of flexible load. The demand for long-duration or seasonal storage capacity does not decline much from this short-term demand flexibility, but the need for its expensive charge and discharge capacity is reduced as the short-term demand flexibility can help to reduce the maximum instantaneous production surplus and shortfall peaks that might occur.</p><p>Increasing the flexibility window to 24 hours only moderately improves the efficiency and reduces the cost of the system:</p>
<p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;"></th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">20%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">40%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">60%</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Load (total / avg / peak)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation (total / avg / peak)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">116.4TWh / 13.3GW / 48.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">114.3TWh / 13.1GW / 47.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">111.6TWh / 12.8GW / 46.7GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">109.8TWh / 12.6GW / 45.8GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation PV / ONW / OFFW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.4% / 60.6% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">38.9% / 61.1% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">37.8% / 62.2% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">36.3% / 63.7% / 0.0%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Annual Cost / Cost per MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.5B€ / 85.3 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.0B€ / 80.8 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.7B€ / 77.5 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.5B€ / 75.7 €/MWh</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">System Efficiency</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">85.5%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">87.1%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">89.2%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">90.6%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Surplus</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.5%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.0%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0.7%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0.7%</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Storage contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">28.2TWh (28.4%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">20.1TWh (20.2%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">15.1TWh (15.2%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">12.6TWh (12.7%)</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Power</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.6GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">3.3GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0.5GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Capacity / Duration</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.9GWh / 5.2h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">10.8GWh / 3.3h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0.6GWh / 1.1h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0GWh / 0h</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS contribution</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.5TWh (9.5%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">2.4TWh (2.4%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0.1TWh (0.1%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">0TWh (0%)</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Power (charge/ discharge)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.1GW / 11.0GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">13.4GW / 10.3GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">12.2GW / 9.6GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11.0GW / 8.8GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Capacity / Duration</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5489.6GWh / 500h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5448.4GWh / 526h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5412.6GWh / 564h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5290.0GWh / 603h</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18.8TWh (18.9%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">17.8TWh (17.9%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">15.0TWh (15.1%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">12.6TWh (12.7%)</td>
</tr>
</tbody>
</table><p></p>
<p>Even if we assume that the system were to accomodate for 60% of the load to be flexible anywhere within a 24h range, only a small part of that potential is actually used. Most of the load can still be satisfied close to the time when it is nominally requested:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpWH2v-Y3dwajSRhwwJHvOBzl2wd8uuyVi2Q8Ua98FZ0vOUBe7nuwc2p3sWsU8uolCJyFZZVuzp-x3jvteqmsMEAUnNf7D9nqe682OgdQ9xGsrOT6hjcVwmckHRgIXm0zdPrNth5pQLStNjpIM45QWdPRNUAEe2OJT0Fh57N9Aa-Cyn6XJmYEL7RfF8ADt/s580/histo.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="434" data-original-width="580" height="372" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpWH2v-Y3dwajSRhwwJHvOBzl2wd8uuyVi2Q8Ua98FZ0vOUBe7nuwc2p3sWsU8uolCJyFZZVuzp-x3jvteqmsMEAUnNf7D9nqe682OgdQ9xGsrOT6hjcVwmckHRgIXm0zdPrNth5pQLStNjpIM45QWdPRNUAEe2OJT0Fh57N9Aa-Cyn6XJmYEL7RfF8ADt/w498-h372/histo.png" width="498" /></a></div><p>This simulation assumes a lossless distribution network without capacity constraints. In reality most power grids face severe limitation and will require significant capacity increase to keep up with the expected growth in load. We should take advantage of the inherent flexibility for most this new load. This would be significantly cheaper than building new transmission capacity. Or in other words: silicon and software are generally much cheaper than copper!</p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-47787690813230078002023-07-15T17:00:00.002+02:002023-07-15T17:48:40.646+02:00Linear Programming with Energy Data<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY88ItLsMwllhaZshV8H5IaaGPEnIIgTuDSvk7SPD2tPswEEBnxZciruHOBDiqPPHVLdRKwUr3zp5Rdf30h62d_IXy30Dme64xyxL169Nri1xYtHXfMzeJ2Ha7_Q1l5GAspSFskEQEBV4MIscY2qCtdUS3XuT35o9xUXyFE51iIM7HPpkaIefVycZsz_Ux/s1535/2016-03-04%2019.40.32.jpg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1535" data-original-width="1024" height="509" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY88ItLsMwllhaZshV8H5IaaGPEnIIgTuDSvk7SPD2tPswEEBnxZciruHOBDiqPPHVLdRKwUr3zp5Rdf30h62d_IXy30Dme64xyxL169Nri1xYtHXfMzeJ2Ha7_Q1l5GAspSFskEQEBV4MIscY2qCtdUS3XuT35o9xUXyFE51iIM7HPpkaIefVycZsz_Ux/w338-h509/2016-03-04%2019.40.32.jpg" width="338" /></a></div>In the <a href="https://blog.kugelfish.com/2023/07/solar-wind-storage-optimal-resource.html" target="_blank">previous post</a>, we made some back of the envelope estimation of what an optimal solar, wind & storage based power grid could look like. For the analysis we were using a standard linear programming model using open-data from the <a href="https://blog.kugelfish.com/2023/01/energy-data-analysis-with-pandas.html" target="_blank">ENTSO-E transparency platform</a> as well as a Python based <a href="https://en.wikipedia.org/wiki/Linear_programming" target="_blank">linear programming</a> (LP) framework.<p></p><p>For the LP framework we chose <a href="https://pypi.org/project/PuLP/" target="_blank">PuLP</a> for its beginner friendly fluent and natural sounding model definition. See <a href="https://machinelearninggeek.com/solving-linear-programming-using-python-pulp/" target="_blank">this tutorial</a> for a more detailed introduction into LP with PuLP or this this <a href="https://realpython.com/linear-programming-python/" target="_blank">comparison of some popular Python LP frameworks</a>. Understanding and debugging the output of an LP solver is non-obvious for casual users. Hence, being able to translate the mathematical model into what looks like natural python code can help to prevent mistakes during model setup.</p><p>The PuLP framework can interface with different external solver backends out of which the <a href="https://github.com/coin-or/Cbc" target="_blank">CBC solver</a> is packaged with the PuLP distribution.</p><p>As input data, we can get time-series data with hourly granularity for a given year for the total load (electricity consumption) as well as the total forecasted production potential for solar, onshore and offshore wind with an hourly granularity as well. This grid-level granularity data essentially represents the demand and production capacity of a given grid-zone (typically a country) in aggregate.</p><p>The key question to ask the model is by how much would the existing capacity for each of the renewable source categories need to be scaled up such that it could satisfy the current demand in combination with an assumed storage system.</p><p>The objective of the model is to minimize the annual cost which can satisfy the operational balancing constraints during each hourly interval.</p><p>Since the forecasting data provides estimates of electricity production volume for each hour, we are using the <a href="https://en.wikipedia.org/wiki/Cost_of_electricity_by_source" target="_blank">levelized cost of electricity</a> in Euro per MWh for each of the source types to determine to total cost of electricity produced, without knowing what would be installed generation capacity required. We are making the assumption that the current production profile or a country or region could be scaled up linearly while maintaining the same hourly distribution of generation over time.</p><p>For the storage model we are going to estimate the charge & discharge power and storage capacity respectively that would need to be installed to satisfy the constraints and are using and capital and operating cost estimate to translate these installed capacities into an annualised cost assuming a 3% interest rate over 15 or 25 year amortisation periods for the capital cost. The storage system is broken up into lithium-ion battery inspired short-duration storage (SDS) and H2 inspired long-duration storage (LDS) respectively. Which means the SDS systems has a single, shared charge and discharge capacity, while the LDS can be charged and discharged independently using different power conversion systems.</p><p>Which all this results in the following hard-coded cost assumptions:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(248, 248, 248); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">Sources <span style="color: #666666;">=</span> enum<span style="color: #666666;">.</span>Enum(<span style="color: #bb4444;">'Sources'</span>, [<span style="color: #bb4444;">'PV'</span>, <span style="color: #bb4444;">'ONW'</span>, <span style="color: #bb4444;">'OFFW'</span>])
StorageConfig <span style="color: #666666;">=</span> namedtuple(<span style="color: #bb4444;">'StorageConfig'</span>, <span style="color: #bb4444;">'name cp_cost, dp_cost, c_loss, d_loss, capacity_cost'</span>)
<span style="color: #008800; font-style: italic;"># LCOE for renewable energy sources in Euro per MWh</span>
<span style="color: #008800; font-style: italic;"># (https://en.wikipedia.org/wiki/Cost_of_electricity_by_source and</span>
<span style="color: #008800; font-style: italic;"># https://www.ise.fraunhofer.de/en/publications/studies/cost-of-electricity.html)</span>
lcoe <span style="color: #666666;">=</span> {
Sources<span style="color: #666666;">.</span>PV : <span style="color: #666666;">55</span>,
Sources<span style="color: #666666;">.</span>ONW : <span style="color: #666666;">60</span>,
Sources<span style="color: #666666;">.</span>OFFW : <span style="color: #666666;">90</span>
}
<span style="color: #008800; font-style: italic;"># Annualised cost for storage technologies in Euro per MW / MWh respectively</span>
<span style="color: #008800; font-style: italic;"># ( https://iopscience.iop.org/article/10.1088/1748-9326/ac4dc8#erlac4dc8s2-2</span>
<span style="color: #008800; font-style: italic;"># / Appendix C)</span>
inverter_cost <span style="color: #666666;">=</span> amount<span style="color: #666666;">.</span>calculate_amortization_amount(<span style="color: #666666;">100000</span>, <span style="color: #666666;">0.03</span>, <span style="color: #666666;">15</span>)
battery_cost <span style="color: #666666;">=</span> amount<span style="color: #666666;">.</span>calculate_amortization_amount(<span style="color: #666666;">125000</span>, <span style="color: #666666;">0.03</span>, <span style="color: #666666;">15</span>)
sds <span style="color: #666666;">=</span> StorageConfig(<span style="color: #bb4444;">'Li-ion'</span>, inverter_cost<span style="color: #666666;">/2</span>, inverter_cost<span style="color: #666666;">/2</span>, <span style="color: #666666;">0.05</span>, <span style="color: #666666;">0.05</span>,
battery_cost)
h2_electrolyzer_cost <span style="color: #666666;">=</span> amount<span style="color: #666666;">.</span>calculate_amortization_amount(<span style="color: #666666;">450000</span>, <span style="color: #666666;">0.03</span>, <span style="color: #666666;">25</span>) <span style="color: #666666;">+</span> <span style="color: #666666;">9000</span>
h2_ccgt_cost <span style="color: #666666;">=</span> amount<span style="color: #666666;">.</span>calculate_amortization_amount(<span style="color: #666666;">750000</span>, <span style="color: #666666;">0.03</span>, <span style="color: #666666;">25</span>) <span style="color: #666666;">+</span> <span style="color: #666666;">15000</span>
h2_storage_cost <span style="color: #666666;">=</span> amount<span style="color: #666666;">.</span>calculate_amortization_amount(<span style="color: #666666;">2000</span>, <span style="color: #666666;">0.03</span>, <span style="color: #666666;">25</span>)
lds <span style="color: #666666;">=</span> StorageConfig(<span style="color: #bb4444;">'H2 P2X2P'</span>, h2_electrolyzer_cost, h2_ccgt_cost, <span style="color: #666666;">0.2</span>, <span style="color: #666666;">0.4</span>,
h2_storage_cost)</pre></div><p>To express the cost function requires the definition of a few global variables for things like the global capacity and cost allocations, as well as time-series indexed variables to express constraints for each of the 8760 hourly intervals in a year.</p><p>The system constraints are relatively simple: the power balance requires that for each interval, the sum of all power generation and consumption needs to balance out. For each storage system the SOC constraint requires that the state of charge in the next interval is the current state of charge adjusted for any charge and discharge - taking into account losses.</p><p>At any time, capacity constraints need to be respected, i.e. state of charge cannot exceed the storage capacity or charge and discarge power cannot exceed the respective power capacity limits.</p><p>Storage should be a closed, cyclical system over the year which is arbitrarily enforced by making the state of charge after the last interval to be the same as the original one.</p><p>Finally the availability or coverage constraint defines what percentage of overall demand must be satisfied by the system.</p>
<!--HTML generated using hilite.me--><div style="background: rgb(248, 248, 248); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">model <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpProblem(<span style="color: #bb4444;">'RE_deployment'</span>, pulp<span style="color: #666666;">.</span>LpMinimize)
source_factors <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable<span style="color: #666666;">.</span>dicts(<span style="color: #bb4444;">'source_factors'</span>, <span style="color: #aa22ff;">list</span>(Sources), <span style="color: #666666;">0</span>)
sds_power <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable(<span style="color: #bb4444;">'sds_power'</span>, <span style="color: #666666;">0</span>)
sds_capacity <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable(<span style="color: #bb4444;">'sds_capacity'</span>, <span style="color: #666666;">0</span>)
lds_c_power <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable(<span style="color: #bb4444;">'lds_c_power'</span>, <span style="color: #666666;">0</span>)
lds_d_power <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable(<span style="color: #bb4444;">'lds_d_power'</span>, <span style="color: #666666;">0</span>)
lds_capacity <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable(<span style="color: #bb4444;">'lds_capacity'</span>, <span style="color: #666666;">0</span>)
total_curtailment <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable(<span style="color: #bb4444;">'total_curtailment'</span>, <span style="color: #666666;">0</span>)
total_shortfall <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable(<span style="color: #bb4444;">'total_shortfall'</span>, <span style="color: #666666;">0</span>)
<span style="color: #008800; font-style: italic;"># Expressing the cost function to be minimised as the sum of all annual costs or energy production</span>
<span style="color: #008800; font-style: italic;"># from all types of renewable sources as well as the storage capacity costs.</span>
<span style="color: #008800; font-style: italic;"># Note the small bonus term for curtailed surplus production to avoid storage over-utilization</span>
<span style="color: #008800; font-style: italic;"># artifacts.</span>
model <span style="color: #666666;">+=</span> (pulp<span style="color: #666666;">.</span>lpSum(source_factors[src] <span style="color: #666666;">*</span> source_total[src] <span style="color: #666666;">*</span> lcoe[src] <span style="color: #aa22ff; font-weight: bold;">for</span> src <span style="color: #aa22ff; font-weight: bold;">in</span> Sources)
<span style="color: #666666;">+</span> sds_power <span style="color: #666666;">*</span> (sds<span style="color: #666666;">.</span>cp_cost <span style="color: #666666;">+</span> sds<span style="color: #666666;">.</span>dp_cost) <span style="color: #666666;">+</span> sds_capacity <span style="color: #666666;">*</span> sds<span style="color: #666666;">.</span>capacity_cost
<span style="color: #666666;">+</span> lds_c_power <span style="color: #666666;">*</span> lds<span style="color: #666666;">.</span>cp_cost <span style="color: #666666;">+</span> lds_d_power <span style="color: #666666;">*</span> lds<span style="color: #666666;">.</span>dp_cost <span style="color: #666666;">+</span> lds_capacity <span style="color: #666666;">*</span> lds<span style="color: #666666;">.</span>capacity_cost
<span style="color: #666666;">-</span> <span style="color: #666666;">10e-6</span> <span style="color: #666666;">*</span> total_curtailment
), <span style="color: #bb4444;">'minimize yearly cost'</span>
<span style="color: #008800; font-style: italic;"># enumerate all the value of the time-series index</span>
hours <span style="color: #666666;">=</span> <span style="color: #aa22ff;">list</span>(<span style="color: #aa22ff;">range</span>(ts_size))
<span style="color: #008800; font-style: italic;"># Create time-series indexed variables</span>
sds_charge <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable<span style="color: #666666;">.</span>dicts(<span style="color: #bb4444;">'sds_charge'</span>, hours, <span style="color: #666666;">0</span>)
sds_discharge <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable<span style="color: #666666;">.</span>dicts(<span style="color: #bb4444;">'sds_discharge'</span>, hours, <span style="color: #666666;">0</span>)
sds_soc <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable<span style="color: #666666;">.</span>dicts(<span style="color: #bb4444;">'sds_soc'</span>, <span style="color: #aa22ff;">list</span>(<span style="color: #aa22ff;">range</span>(ts_size <span style="color: #666666;">+</span> <span style="color: #666666;">1</span>)), <span style="color: #666666;">0</span>)
lds_charge <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable<span style="color: #666666;">.</span>dicts(<span style="color: #bb4444;">'lds_charge'</span>, hours, <span style="color: #666666;">0</span>)
lds_discharge <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable<span style="color: #666666;">.</span>dicts(<span style="color: #bb4444;">'lds_discharge'</span>, hours, <span style="color: #666666;">0</span>)
lds_soc <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable<span style="color: #666666;">.</span>dicts(<span style="color: #bb4444;">'lds_soc'</span>, <span style="color: #aa22ff;">list</span>(<span style="color: #aa22ff;">range</span>(ts_size <span style="color: #666666;">+</span> <span style="color: #666666;">1</span>)), <span style="color: #666666;">0</span>)
curtailment <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable<span style="color: #666666;">.</span>dicts(<span style="color: #bb4444;">'curtailment'</span>, hours, <span style="color: #666666;">0</span>)
shortfall <span style="color: #666666;">=</span> pulp<span style="color: #666666;">.</span>LpVariable<span style="color: #666666;">.</span>dicts(<span style="color: #bb4444;">'shortfall'</span>, hours, <span style="color: #666666;">0</span>)
<span style="color: #aa22ff; font-weight: bold;">for</span> i <span style="color: #aa22ff; font-weight: bold;">in</span> hours:
<span style="color: #008800; font-style: italic;"># primary power balance constraint</span>
model <span style="color: #666666;">+=</span> (pulp<span style="color: #666666;">.</span>lpSum(source_factors[src] <span style="color: #666666;">*</span> source_ts[src][i] <span style="color: #aa22ff; font-weight: bold;">for</span> src <span style="color: #aa22ff; font-weight: bold;">in</span> Sources)
<span style="color: #666666;">+</span> sds_discharge[i] <span style="color: #666666;">-</span> sds_charge[i]
<span style="color: #666666;">+</span> lds_discharge[i] <span style="color: #666666;">-</span> lds_charge[i]
<span style="color: #666666;">-</span> curtailment[i] <span style="color: #666666;">+</span> shortfall[i] <span style="color: #666666;">==</span> load_ts[i])
<span style="color: #008800; font-style: italic;"># storage charge/discharge power limit</span>
model <span style="color: #666666;">+=</span> (sds_discharge[i] <span style="color: #666666;">+</span> sds_charge[i] <span style="color: #666666;"><=</span> sds_power)
model <span style="color: #666666;">+=</span> (lds_charge[i] <span style="color: #666666;"><=</span> lds_c_power)
model <span style="color: #666666;">+=</span> (lds_discharge[i] <span style="color: #666666;"><=</span> lds_d_power)
<span style="color: #008800; font-style: italic;"># State of charge constraints</span>
model <span style="color: #666666;">+=</span> (sds_soc[i] <span style="color: #666666;">-</span> sds_discharge[i] <span style="color: #666666;">*</span> (<span style="color: #666666;">1</span> <span style="color: #666666;">+</span> sds<span style="color: #666666;">.</span>d_loss) <span style="color: #666666;">+</span> sds_charge[i] <span style="color: #666666;">*</span> (<span style="color: #666666;">1</span> <span style="color: #666666;">-</span> sds<span style="color: #666666;">.</span>c_loss) <span style="color: #666666;">==</span> sds_soc[i <span style="color: #666666;">+</span> <span style="color: #666666;">1</span>])
model <span style="color: #666666;">+=</span> (lds_soc[i] <span style="color: #666666;">-</span> lds_discharge[i] <span style="color: #666666;">*</span> (<span style="color: #666666;">1</span> <span style="color: #666666;">+</span> lds<span style="color: #666666;">.</span>d_loss) <span style="color: #666666;">+</span> lds_charge[i] <span style="color: #666666;">*</span> (<span style="color: #666666;">1</span> <span style="color: #666666;">-</span> lds<span style="color: #666666;">.</span>c_loss) <span style="color: #666666;">==</span> lds_soc[i <span style="color: #666666;">+</span> <span style="color: #666666;">1</span>])
<span style="color: #008800; font-style: italic;"># State of charge cannot exceed storage capacity</span>
model <span style="color: #666666;">+=</span> (sds_soc[i] <span style="color: #666666;"><=</span> sds_capacity)
model <span style="color: #666666;">+=</span> (lds_soc[i] <span style="color: #666666;"><=</span> lds_capacity)
<span style="color: #008800; font-style: italic;"># Yearly cycle should represent a closed loop</span>
model <span style="color: #666666;">+=</span> (sds_soc[<span style="color: #666666;">0</span>] <span style="color: #666666;">==</span> sds_soc[ts_size])
model <span style="color: #666666;">+=</span> (lds_soc[<span style="color: #666666;">0</span>] <span style="color: #666666;">==</span> lds_soc[ts_size])
<span style="color: #008800; font-style: italic;"># constraint for availability. I.e. how much of the demand should be met by this system.</span>
model <span style="color: #666666;">+=</span> (pulp<span style="color: #666666;">.</span>lpSum(shortfall[i] <span style="color: #aa22ff; font-weight: bold;">for</span> i <span style="color: #aa22ff; font-weight: bold;">in</span> hours) <span style="color: #666666;"><=</span> load_total <span style="color: #666666;">*</span> (<span style="color: #666666;">1</span> <span style="color: #666666;">-</span> availability_factor))
<span style="color: #008800; font-style: italic;"># Convenience rollup from ts into total values for curtailment and shortfall</span>
model <span style="color: #666666;">+=</span> (pulp<span style="color: #666666;">.</span>lpSum(curtailment[i] <span style="color: #aa22ff; font-weight: bold;">for</span> i <span style="color: #aa22ff; font-weight: bold;">in</span> hours) <span style="color: #666666;">-</span> total_curtailment <span style="color: #666666;">==</span> <span style="color: #666666;">0</span>)
model <span style="color: #666666;">+=</span> (pulp<span style="color: #666666;">.</span>lpSum(shortfall[i] <span style="color: #aa22ff; font-weight: bold;">for</span> i <span style="color: #aa22ff; font-weight: bold;">in</span> hours) <span style="color: #666666;">-</span> total_shortfall <span style="color: #666666;">==</span> <span style="color: #666666;">0</span>)
<span style="color: #008800; font-style: italic;"># solve</span>
results <span style="color: #666666;">=</span> model<span style="color: #666666;">.</span>solve()
</pre></div>
<p>The load and production forecast data which are used as the constant parameters in the linear inequality constraints for each interval in the time-series can be obtained from ENTSO-E API. We resample the time series from potentially 15min granularity into hourly to facilitate an easy conversion between power levels and energy (1 MW for 1h -> 1MWh).</p><p>
<!--HTML generated using hilite.me--></p><div style="background: rgb(248, 248, 248); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">client <span style="color: #666666;">=</span> EntsoePandasClient(api_key<span style="color: #666666;">=</span><span style="color: #bb4444;">'<api-key>'</span>)
country <span style="color: #666666;">=</span> <span style="color: #bb4444;">'NL'</span>
start <span style="color: #666666;">=</span> pd<span style="color: #666666;">.</span>Timestamp(<span style="color: #bb4444;">'20220401'</span>, tz<span style="color: #666666;">=</span><span style="color: #bb4444;">'Europe/Berlin'</span>)
end <span style="color: #666666;">=</span> pd<span style="color: #666666;">.</span>Timestamp(<span style="color: #bb4444;">'20230401'</span>, tz<span style="color: #666666;">=</span><span style="color: #bb4444;">'Europe/Berlin'</span>)
forecast <span style="color: #666666;">=</span> client<span style="color: #666666;">.</span>query_wind_and_solar_forecast(country, start<span style="color: #666666;">=</span>start, end<span style="color: #666666;">=</span>end)
load <span style="color: #666666;">=</span> client<span style="color: #666666;">.</span>query_load(country, start<span style="color: #666666;">=</span>start, end<span style="color: #666666;">=</span>end)
<span style="color: #008800; font-style: italic;"># Align with hourly resolution for easy conversion from power to energy (MH / MWh)</span>
forecast <span style="color: #666666;">=</span> forecast<span style="color: #666666;">.</span>resample(<span style="color: #bb4444;">'H'</span>)<span style="color: #666666;">.</span>agg(<span style="color: #bb4444;">'mean'</span>)<span style="color: #666666;">.</span>fillna(<span style="color: #666666;">0</span>)
load <span style="color: #666666;">=</span> load<span style="color: #666666;">.</span>resample(<span style="color: #bb4444;">'H'</span>)<span style="color: #666666;">.</span>agg(<span style="color: #bb4444;">'mean'</span>)<span style="color: #666666;">.</span>fillna(<span style="color: #666666;">0</span>)
load_ts <span style="color: #666666;">=</span> load[<span style="color: #bb4444;">'Actual Load'</span>]
load_total <span style="color: #666666;">=</span> load_ts<span style="color: #666666;">.</span>sum()
<span style="color: #008800; font-style: italic;"># create an empty/zero-valued time-series for missing source types</span>
forecast[<span style="color: #bb4444;">'default'</span>] <span style="color: #666666;">=</span> <span style="color: #666666;">0.0</span>
source_ts <span style="color: #666666;">=</span> {
Sources<span style="color: #666666;">.</span>PV: forecast<span style="color: #666666;">.</span>get(<span style="color: #bb4444;">'Solar'</span>, forecast[<span style="color: #bb4444;">'default'</span>]),
Sources<span style="color: #666666;">.</span>ONW: forecast<span style="color: #666666;">.</span>get(<span style="color: #bb4444;">'Wind Onshore'</span>, forecast[<span style="color: #bb4444;">'default'</span>]),
Sources<span style="color: #666666;">.</span>OFFW: forecast<span style="color: #666666;">.</span>get(<span style="color: #bb4444;">'Wind Offshore'</span>, forecast[<span style="color: #bb4444;">'default'</span>])
}
source_total <span style="color: #666666;">=</span><span style="color: #aa22ff;">dict</span>([(source, source_ts[source]<span style="color: #666666;">.</span>sum()) <span style="color: #aa22ff; font-weight: bold;">for</span> source <span style="color: #aa22ff; font-weight: bold;">in</span> Sources])
ts_size <span style="color: #666666;">=</span> <span style="color: #aa22ff;">min</span>([load_ts<span style="color: #666666;">.</span>size] <span style="color: #666666;">+</span> [ts<span style="color: #666666;">.</span>size <span style="color: #aa22ff; font-weight: bold;">for</span> ts <span style="color: #aa22ff; font-weight: bold;">in</span> source_ts<span style="color: #666666;">.</span>values()])
</pre></div>
<p></p><p><br /></p><p></p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-1511871967955758952023-07-08T19:47:00.026+02:002023-07-15T17:25:19.146+02:00Solar, Wind & Storage - optimal resource allocation<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-UBEi5vk_8-k9Rd95K4Lstv0-CIGkn8nq_ylzEOF9a1jkjKLeRmDhrxpQZJ23il2vzszm-Sw8QeXnzeQjufL376IMukobGawiGsSs9F4co71uIW7ilkjs-C-tFEJSi3FBsouFoA34ax20jJWLVMylKLtT8w627eqnlTWQuvdxpgfl14ZnCe6BjhSMGyS9/s2252/IMG_0371.JPG" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="2252" data-original-width="1647" height="354" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-UBEi5vk_8-k9Rd95K4Lstv0-CIGkn8nq_ylzEOF9a1jkjKLeRmDhrxpQZJ23il2vzszm-Sw8QeXnzeQjufL376IMukobGawiGsSs9F4co71uIW7ilkjs-C-tFEJSi3FBsouFoA34ax20jJWLVMylKLtT8w627eqnlTWQuvdxpgfl14ZnCe6BjhSMGyS9/w259-h354/IMG_0371.JPG" width="259" /></a></div>As wind and solar are reaching a double-digit share of production in many power grids, the focus is increasingly shifting towards adding storage. Large consumers like Google are pushing their suppliers towards <a href="https://sustainability.google/progress/energy/" target="_blank">providing low carbon energy 24/7 all year round</a>. The same goal is also captured well by the catchy new term of "<a href="https://www.50hertz.com/xspProxy/api/staticfiles/50hertz-client/dokumente/unternehmen/partnerschaften/gr%C3%BCnegrundlast.pdf" target="_blank">Grüne Grundlast</a>" (green baseload) , despite being slightly misleading as the goal should not be constant production when nobody needs it, but production which matches the load at all times.<p></p><p>Using national power grid data from the <a href="https://blog.kugelfish.com/2023/01/energy-data-analysis-with-pandas.html" target="_blank">ENTSO-E transparency platform</a>, some very rough and hand-wavy cost assumptions and a <a href="https://pypi.org/project/PuLP/" target="_blank">standard Python LP solver</a>, we can do a back of the envelope estimation of what it would take to satisfy todays electricity needs of some European countries using an extrapolation of their current solar and wind production in combination with a fictitious storage pool.</p><p> <a href="https://en.wikipedia.org/wiki/Linear_programming" target="_blank">Linear Programming</a> is a very common optimisation technique used in operations research and economic modelling which can be applied as long as a problem can be expressed as minimisation of a linear cost function und the constraint of a series of linear inequalities.</p><p>For the storage system, we are assuming a combination of "short duration storage" (SDS) inspired by current lithium-ion <a href="https://en.wikipedia.org/wiki/Battery_storage_power_station" target="_blank">battery storage plants</a> and "long duration storage" (LDS) inspired by a hydrogen based storage plant consisting of electrolyzers to convert electricity into hydrogen, <a href="https://en.wikipedia.org/wiki/Underground_hydrogen_storage" target="_blank">storage in underground caverns</a> and conversion back into electricity using conventional <a href="https://en.wikipedia.org/wiki/Combined_cycle_power_plant" target="_blank">combined-cycle gas turbine plants</a>. The cost assumptions for this configuration are taken from <a href="https://iopscience.iop.org/article/10.1088/1748-9326/ac4dc8#erlac4dc8s2-2" target="_blank">this paper</a>, which also helped to inspire and validate the formulation of the LP model - which is described in more detail in <a href="https://blog.kugelfish.com/2023/07/linear-programming-with-energy-data.html" target="_blank">this following post</a>. The capital costs are annualised using a 3% interest rates and added to the annual operating cost estimates.</p><p>For the cost of solar and wind generation, we are using a LCOE cost estimates in Euro per MWh from <a href="https://en.wikipedia.org/wiki/Cost_of_electricity_by_source" target="_blank">Wikipedia</a>, which are based on this 2021 <a href="https://www.ise.fraunhofer.de/en/publications/studies/cost-of-electricity.html">report</a>. Given the wide range of values, we are using rough median values of 55, 60 and 90 Euro/MWh for solar, onshore wind and offshore wind respectively.</p><p>As an example for the hourly load and generation profiles, we are using the Netherlands, which has significant production forecast data for all the 3 types of renewable sources we are considering: solar, onshore wind and offshore wind.</p><p>We are running the LP model for different target levels of coverage or system availability factors for this fictitious production system. This represents the ability to satisfy 80 to 100% of the demand from solar, wind & storage alone, using the actual load & generation data from spring 2022 to spring 2023. A coverage factor of 80% implies that the simulated system would only satisfy 80% of the load, with the remaining 20% coming from other sources - import or other sources of generation.</p>
<p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;"></th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">80%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">90%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">95%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">100%</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Load (total / avg / peak)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation (total / avg / peak)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">87.2TWh / 10.0GW / 36.8GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">101.0TWh / 11.5GW / 42.5GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">108.1TWh / 12.4GW / 45.6GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">116.4TWh / 13.3GW / 48.9GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation PV / ONW / OFFW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">43.1% / 56.9% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">40.8% / 59.2% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">41.3% / 58.7% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.4% / 60.6% / 0.0%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Annual Cost / Cost per MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5.8B€ / 73.2 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">6.9B€ / 77.4 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.6B€ / 80.2 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.5B€ / 85.3 €/MWh</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">System Efficiency</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">91.4%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">88.8%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">87.5%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">85.5%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Surplus</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.7%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.6%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.5%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1.5%</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Storage contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">16.4TWh (16.5%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">22.3TWh (22.4%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">25.9TWh (26.0%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">28.2TWh (28.4%)</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Power</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.2GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.4GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.6GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Capacity / Duration</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">44.3GWh / 5.4h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">48.9GWh / 5.5h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">51.4GWh / 5.4h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.9GWh / 5.2h</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS contribution</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.8TWh (9.8%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">10.9TWh (11.0%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11.8TWh (11.8%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.5TWh (9.5%)</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Power (charge/ discharge)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">6.6GW / 1.5GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.4GW / 3.5GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11.0GW / 4.4GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.1GW / 11.0GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Capacity / Duration</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">1273.2GWh / 855h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">2310.2GWh / 662h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">3767.8GWh / 847h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5489.6GWh / 500h</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">6.7TWh (6.7%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11.4TWh (11.4%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.2TWh (14.2%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18.8TWh (18.9%)</td>
</tr>
</tbody>
</table><p></p>
<p>The resulting optimal allocation of annual cost into the choices of resources is as follows, resulting in an average per MWh production cost between 73 and 85 Euro per MWh depending on the achieved coverage/availability:</p><p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigItp9AVlB8mmfPVQZdGGFBklqGJb03n9t4iqxTtWi-h29o4VAVegSYijb3BQg-hGj45t6W243ImwpNpy4_AHGdl0a8owWUsgPy9OLd3xU47x_WvilM8hcaWl8OrkLRMg50QE9efSNQY1EfLr9y92e-iMIYOaaSs2ClIiysV5nnXF8J21bSt6c__4-RYP1/s650/allocation.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="606" data-original-width="650" height="596" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigItp9AVlB8mmfPVQZdGGFBklqGJb03n9t4iqxTtWi-h29o4VAVegSYijb3BQg-hGj45t6W243ImwpNpy4_AHGdl0a8owWUsgPy9OLd3xU47x_WvilM8hcaWl8OrkLRMg50QE9efSNQY1EfLr9y92e-iMIYOaaSs2ClIiysV5nnXF8J21bSt6c__4-RYP1/w640-h596/allocation.png" width="640" /></a></div>The resulting system is surprisingly well-balanced, with an energy generation surplus (curtailment or export) of only 1.5-2%. While solar is slightly cheaper than wind per MWh generated, the optimisation seems to favor wind due to a more even match of supply and demand, reducing the need for even more expensive storage. Offshore wind is excluded from most solutions, presumably due to being more expensive than onshore wind and not offering a sufficiently complementary production profile.<div><br /></div><div>For the storage model, the battery based SDS is very efficient (>90% round-trip efficiency) with expensive storage cost but a lower additional per power capacity cost compared to hydrogen LDS which requires very expensive conversion equipment. For the optimal solution overall storage need is driving the dimensioning of the LDS system while the SDS systems seems to mostly serve to supplement peak power capacity as illustrated by a energy flow graph for a few days in April 2022 in the simulated model:</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii-pyQwUagZStYtMU-RSV3u7DGUVD5hX3UKsmdclEmiz_J_XF0TzjjS8OhghFS2qr4B6Ah66wAjTtd8W2Jgtm4R2LsRtfKfUuRNW-DF7VGt6ofTzvxOP23kZr5A57XPfbo_D9g3_MRPvxUm60WTM__bKLjp4lnB9uyCYcAWX_XGQH9ZxGlSGCZT6fT_3Oa/s543/fig.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="438" data-original-width="543" height="516" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii-pyQwUagZStYtMU-RSV3u7DGUVD5hX3UKsmdclEmiz_J_XF0TzjjS8OhghFS2qr4B6Ah66wAjTtd8W2Jgtm4R2LsRtfKfUuRNW-DF7VGt6ofTzvxOP23kZr5A57XPfbo_D9g3_MRPvxUm60WTM__bKLjp4lnB9uyCYcAWX_XGQH9ZxGlSGCZT6fT_3Oa/w640-h516/fig.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">Despite the much lower efficiency, the H2 storage system is also used heavily for short-term intraday turnarounds, where one might have expected the more efficient battery SDS to do most of the heavy lifting. The resulting optimal SDS storage capacity remains relatively small about 5.5 hours only. In a way it seems that the SDS battery system mostly acts as a short-term power-capacity booster to better absorb the sharp solar peaks. The model has no penalty term for ramping, turnaround or usage based degradation, leading to some arbitrary oscillations which likely would be discouraged in a real system.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">The model seems to be mostly driven by a trade-off between low solar & wind productions costs and low hydrogen storage cost while minimising the need for expensive power capacity to transfer between the two. The low energy efficiency of the H2 storage system does not seem to matter much, if the alternative would be more overproduction and curtailment. Or in other words, the optimal configuration is determined by the needs for the worst-case situation and for all the other times, efficiency matters very little. While 5.5 TWh of H2 storage might seem like a lot, <a href="https://www.hyuspre.eu/wp-content/uploads/2022/06/HyUSPRe_D1.3_Hydrogen-storage-potential-of-existing-European-gas-storage-sites_2022.06.29.pdf" target="_blank">this report claims</a> an existing potential of over 140 TWh H2 underground storage for the Netherlands, mostly in the form depleted natural gas fields.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">By increasing storage capacity cost 2x for batteries and 10x for H2 storage, we can see the optimal allocation shifting towards more overproduction and curtailment. Also there is a significant jump in cost & storage investments necessary to meet the last 5% of demand all the time.</div>
<p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;"></th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">80%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">90%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">95%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">100%</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Load (total / avg / peak)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation (total / avg / peak)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">94.0TWh / 10.7GW / 39.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">111.2TWh / 12.7GW / 47.2GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">124.4TWh / 14.2GW / 52.2GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">126.5TWh / 14.5GW / 52.2GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation PV / ONW / OFFW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">45.6% / 54.4% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">45.2% / 54.8% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">39.4% / 60.6% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">30.4% / 69.6% / 0.0%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Annual Cost / Cost per MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">6.4B€ / 80.8 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.0B€ / 89.7 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.2B€ / 96.8 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">12.2B€ / 122.4 €/MWh</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">System Efficiency</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">84.7%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">80.6%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">76.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">78.7%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Surplus</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">6.3%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.7%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">13.7%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">8.7%</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Storage contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.1TWh (14.1%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">19.6TWh (19.7%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">20.6TWh (20.7%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">24.1TWh (24.2%)</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Power</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">3.3GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5.7GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">4.5GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">4.2GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Capacity / Duration</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">12.4GWh / 3.8h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">24.3GWh / 4.3h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">17.1GWh / 3.8h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">13.3GWh / 3.1h</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS contribution</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">3.3TWh (3.3%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">6.0TWh (6.0%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">4.2TWh (4.2%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">3.2TWh (3.3%)</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Power (charge/ discharge)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11.8GW / 4.3GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">13.6GW / 6.1GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">15.2GW / 8.1GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">13.5GW / 12.9GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Capacity / Duration</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">298.9GWh / 69h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">620.6GWh / 101h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">996.8GWh / 123h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">4322.8GWh / 335h</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">10.8TWh (10.8%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">13.6TWh (13.7%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">16.4TWh (16.5%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">20.8TWh (20.9%)</td></tr></tbody></table><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPgIXgxLCvo8h9viyPhP0FWMDlbu4poF4ATcUn_5Al-xYmz6GyXJtWjo4x1kQYTzOeQU23BbghETWS0ZVJtu7m3-GV5MfaImZ5UthL2c5KQ4Cjj0RfL4GoHaHp6IJS_vqKhlCckRJQK6qSDGgN3hn6qIYKxukJfbjdANqfpY8EvBZgegwohywroGVS-FxQ/s659/allocation.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="595" data-original-width="659" height="578" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPgIXgxLCvo8h9viyPhP0FWMDlbu4poF4ATcUn_5Al-xYmz6GyXJtWjo4x1kQYTzOeQU23BbghETWS0ZVJtu7m3-GV5MfaImZ5UthL2c5KQ4Cjj0RfL4GoHaHp6IJS_vqKhlCckRJQK6qSDGgN3hn6qIYKxukJfbjdANqfpY8EvBZgegwohywroGVS-FxQ/w640-h578/allocation.png" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: left;">To create another extreme scenario, we can peg the production from wind at zero only allowing generation from solar, which generally is the most unevenly distributed both across the day and across the seasons, resulting in a much higher storage requirement:</div>
<p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;"></th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">80%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">90%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">95%</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">100%</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Load (total / avg / peak)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99.6TWh / 11GW / 17.7GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation (total / avg / peak)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">95.8TWh / 10.9GW / 58.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">110.7TWh / 12.7GW / 68.1GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">118.7TWh / 13.6GW / 73.0GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">125.9TWh / 14.4GW / 77.4GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Generation PV / ONW / OFFW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">100.0% / 0.0% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">100.0% / 0.0% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">100.0% / 0.0% / 0.0%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">100.0% / 0.0% / 0.0%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Annual Cost / Cost per MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.9B€ / 99.7 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9.8B€ / 109.0 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">10.8B€ / 113.9 €/MWh</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11.9B€ / 119.2 €/MWh</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">System Efficiency</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">83.2%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">80.9%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">79.7%</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">79.1%</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Surplus</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">3.9%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">3.3%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">3.0%</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">2.9%</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Storage contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">41.6TWh (41.8%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">50.2TWh (50.5%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">57.1TWh (57.3%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">59.7TWh (60.0%)</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Power</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">22.8GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">24.2GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">24.9GW</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">26.3GW</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS Capacity / Duration</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">121.5GWh / 5.3h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">124.8GWh / 5.2h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">126.6GWh / 5.1h</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">133.2GWh / 5.1h</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">SDS contribution</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">29.1TWh (29.3%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">31.3TWh (31.5%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">34.5TWh (34.7%)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">34.3TWh (34.4%)</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Power (charge/ discharge)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14.8GW / 2.7GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">21.4GW / 5.4GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">24.9GW / 6.3GW</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">27.3GW / 10.7GW</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS Capacity / Duration</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11486.5GWh / 4202h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">19563.6GWh / 3625h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">24419.0GWh / 3858h</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">28878.8GWh / 2710h</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">LDS contribution</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">12.5TWh (12.5%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18.9TWh (19.0%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">22.5TWh (22.6%)</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">25.4TWh (25.5%)</td>
</tr>
</tbody>
</table><p></p>
<div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">Because the model is only based on a single year, long-term variability and rare, extreme weather events are not taken into account. However, the biggest weakness of any such model is the sensitivity to assumptions. Even changing them within a reasonable range, can change the resulting configuration significantly.</div><div class="separator" style="clear: both; text-align: left;"><p>For the purpose of this analysis, we are looking at the load and generation capacity of a national power grid in aggregate, completely ignoring the internal capacity constraints of the distribution network. Given that in our retail electricity prices, distribution cost accounts for nearly two thirds of the price should serve as an indication of how flawed this assumption is. It is also not clear to what degree a distributed system where many different stakeholders held together by regulation and market designs, could approximate a globally optimal solution.</p></div><div class="separator" style="clear: both; text-align: left;">We could also argue that modelling today's electricity demand is not as relevant for the future, given the planned electrification of sectors which use fossil fuels to day like transportation, building heat or industrial processes. While we expect the demand to increase significantly, the vast majority of this new demand is also flexible in the order of hours or days, as the needed energy can be buffered more cheaply in the form of heat or stored in generously sized car batteries - as we discussed in <a href="https://blog.kugelfish.com/2023/05/it-is-12-noon-do-you-know-where-you-car.html" target="_blank">this previous post</a>. In other words, the future demand profile will likely to be more closely matched in time to the availability of renewable energy.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">Yet even this very basic back of the envelope analysis shows that with an optimal mix of today's technologies, a 100% renewable grid would be possible for an average generation cost that is less than any fossil fuel source (assuming carbon emission certificates close to 100 Euro). The key would be to deploy as much of storage technology with the cheapest cost of storage (likely H2 - regardless of efficiency or power capacity cost) and complement some of the peak power capacity needs with faster but much smaller storage (likely lithium-ion or similar battery technologies).</div>
Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-90077926363865597582023-06-17T16:38:00.005+02:002023-09-07T21:05:22.840+02:00The Impact of Storage Configuration on Electricity Arbitrage<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJB4MB8iac2owDyV8smaJpfcz5BWdeXq6_7IqQUgdQ_KOg8l2ulRBEKsBNqeqh3eOjCaOrM5fZ43FLSvzRZk6p5bTWGfdxalgFegFFwVCl5sbz0qSGz-rXjq0oh6qizE9MmhWpg4Od1he92W7AYvg00myTBe8ifA6rm5hDvzQ_KYuXbEbnhBwfB7fsdQ/s1183/hoover_dam.JPG" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Hoover dam" border="0" data-original-height="1183" data-original-width="1024" height="373" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJB4MB8iac2owDyV8smaJpfcz5BWdeXq6_7IqQUgdQ_KOg8l2ulRBEKsBNqeqh3eOjCaOrM5fZ43FLSvzRZk6p5bTWGfdxalgFegFFwVCl5sbz0qSGz-rXjq0oh6qizE9MmhWpg4Od1he92W7AYvg00myTBe8ifA6rm5hDvzQ_KYuXbEbnhBwfB7fsdQ/w323-h373/hoover_dam.JPG" width="323" /></a></div>In the <a href="https://blog.kugelfish.com/2023/06/buy-low-sell-high-optimal-electricity.html" target="_blank">previous post,</a> we tried to estimate the opportunity for storage based arbitrage in a variety of European day-ahead electricity markets. For that we have used a "fast & efficient" storage model inspired by <a href="https://en.wikipedia.org/wiki/Battery_storage_power_station">lithium-ion based storage systems</a> with a round-trip cycle efficiency of over 90% and the ability to charge or discharge the entire storage capacity during each interval - i.e. a 1MWh storage with a 1 MW input/output power resulting in a storage duration of 1 hour.<p></p><p>Other storage technologies allow a more independent scaling of the maximum charge/discharge power from the maximum amount of energy the storage can hold, typically at a much lower marginal cost per additional unit of capacity.</p><p>One such example is pumped hydro storage, which is by far the most common form of large-scale electricity storage today. The input output power rating is determined by the configuration of the turbines, generators and pumps which contribute significantly to the cost of the installation. The storage capacity is determined by the volume of the water reservoirs, which can potentially be scaled at much lower cost but are subject to suitable terrain conditions. For example the recently completed <a href="https://en.wikipedia.org/wiki/Nant_de_Drance_Hydropower_Plant" target="_blank">Nant de Drance </a>pumped hydropower plant has a maximum power of 900MW and a storage capacity of 20GWh for a storage duration of about 20 hours. The plant claims a round-trip efficiency of 80%, which is at the very high end for this type of storage which typically has an efficiency of around 70%. </p><p>Another storage technology with potential efficiency above 80% and the ability to scale storage capacity independently of input/output power are <a href="https://en.wikipedia.org/wiki/Flow_battery" target="_blank">redox flow batteries</a>. Other grid scale electricity storage technologies that have shown some potential in pilot installations are for example <a href="https://en.wikipedia.org/wiki/Compressed-air_energy_storage" target="_blank">compressed air storage </a>with efficiencies in the 60-70% range or <a href="https://en.wikipedia.org/wiki/Thermal_energy_storage" target="_blank">molten salt thermal storage</a> with efficiencies probably below 40% unless used for power & heat co-generation.</p><p>Using the day-ahead electricity market data for the Netherlands in 2023, which presented the highest single unit arbitrage potential in the <a href="https://blog.kugelfish.com/2023/06/buy-low-sell-high-optimal-electricity.html" target="_blank">previous post</a>, we can now simulate multiple configurations with different round-trip yields and storage durations:</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfzpYQVdNrhrRrAW4_zYBX8OWgyvcZsunBsgFu_GHi-GLTxVhmVnDjaaALlqZQ5GZid8M4f_2pymgoeIy6FF4O9JOUhhwqF2OFgr7zTLks07LcjJm9z_0TK1vifI9-7ylbYrD9JQoPy7u-U6IIePKpezLKfr-DtXz3rTFbQERu7g_ht-mi80_bXctWwg/s634/arbitrage.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="611" data-original-width="634" height="616" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfzpYQVdNrhrRrAW4_zYBX8OWgyvcZsunBsgFu_GHi-GLTxVhmVnDjaaALlqZQ5GZid8M4f_2pymgoeIy6FF4O9JOUhhwqF2OFgr7zTLks07LcjJm9z_0TK1vifI9-7ylbYrD9JQoPy7u-U6IIePKpezLKfr-DtXz3rTFbQERu7g_ht-mi80_bXctWwg/w640-h616/arbitrage.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">NL 2023</td></tr></tbody></table><p>As expected, we see a drop in arbitrage potential per unit of storage capacity as the capacity increases. While the first unit of storage can match the charge & discharge points to the lowest and highest points in the market respectively, the second unit will need to match the second lowest and second highest points and so on. For optimal sizing on storage arbitrage alone, the per capacity marginal cost would have to decline faster than the arbitrage opportunity per capacity.</p><p>Also as expected, the arbitrage opportunity decreases as the storage efficiency decreases. However the relative decline of the revenue opportunity for the first few hours is less steep for lower efficiency systems. Depending on the relative cost of different storage technologies, it might make sense to combine expensive high-efficiency storage with short duration (1-10 hours) with less efficient but much cheaper long duration storage in the order of 6-24 hours.</p><p>For the market conditions in 2019 with presumably a much smaller difference between short term high and low prices, we can also see an increased sensitivity to efficiency: while at 90% efficiency, the optimisation algorithm could have found 733 profitable "trade" opportunities in 2019, at 40% efficiency there would have been only 145.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDZqXdOYgWj2CMKniswdyhLZ-dNbzkLvspXWWVEj_BhvoCjxD-uWAZh6c9ZevG6U7t5YEaxfz_Nids1r1YAGaQFIiwnAXi524XHZ-UjINfW6TLiz7vAvpw1fhqa5HYnW0xpnFLLkIJSx8O7DsQ5RHntSOq7s5i2Zp99TlREUmLzx93cqBci8IXUHn8og/s631/arbitrage_2019.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="613" data-original-width="631" height="622" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDZqXdOYgWj2CMKniswdyhLZ-dNbzkLvspXWWVEj_BhvoCjxD-uWAZh6c9ZevG6U7t5YEaxfz_Nids1r1YAGaQFIiwnAXi524XHZ-UjINfW6TLiz7vAvpw1fhqa5HYnW0xpnFLLkIJSx8O7DsQ5RHntSOq7s5i2Zp99TlREUmLzx93cqBci8IXUHn8og/w640-h622/arbitrage_2019.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">NL 2019</td></tr></tbody></table><br /><p>For some concrete examples, the above mentioned Nant de Drance pumped hydro storage plant with a power of 900MW, a storage duration of 20h and an efficiency of 80% could in the dutch day-ahead market have generated an arbitrage return of €27,025,200 in 2019 and €274,866,300 in 2022. The very roughly estimated cost for this plant based on the investment of 2 billion CHF a long term utility bond rate of 1.25% over 30 years and an annual operating cost of 2% comes to about 120M CHF per year.</p><p>By comparison, a quick search on the internet shows 20kWh Lithium-ion (LFP) based home storage systems for around €14k (i.e. €700k per MWh) Assuming a 10 year amortisation at current mortgage rates of 3% would yield an annual cost of about €82k for a storage system of 1MW/1MWh vs the €89k arbitrage opportunity in 2022. Current Lithium-ion battery pack prices should be around $100 per kWh (<a href="https://about.bnef.com/blog/battery-pack-prices-fall-to-an-average-of-132-kwh-but-rising-commodity-prices-start-to-bite/" target="_blank">Bloomberg</a>) and the balance of system costs for a single grid-scale battery-electric storage system should be much lower than stacking up 50 home storage systems at retail prices. Assuming a 3x overhead or a rough cost of €300k per MWh amortised over 10 years at 3% would result in an annual cost of €35k vs. an arbitrage opportunity of €89k in 2022 or €10k in 2019.</p><p>While electricity storage costs are far from being cheap, they could under some circumstances be in the range of competing with gas peaker plants which typically set the market price during high demand periods without solar & wind oversupply. If or when this would be the case largely depends on the continued buildout of wind and solar increasing the hours of oversupply, the <a href="https://tradingeconomics.com/commodity/eu-natural-gas" target="_blank">market rate for natural gas</a>, the cost of <a href="https://tradingeconomics.com/commodity/carbon" target="_blank">CO2 emissions certificates</a>, the allocation of government subsidies and the continued cost decline for various storage technologies.</p><p>Besides short-term energy arbitrage there are also other, presumably more lucrative reasons to deploy more storage in power grids today. These range from providing grid <a href="https://en.wikipedia.org/wiki/Ancillary_services_(electric_power)" target="_blank">stability and control</a> to <a href="https://www.swissinfo.ch/eng/politics/swiss-government-plans-hydropower-reserves-and-power-plants/47357234" target="_blank">providing spare capacity</a> to ensure a long-term security of the energy supply.</p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-67844352167704110942023-06-13T23:02:00.006+02:002023-07-14T17:13:47.457+02:00Buy Low, Sell High - Optimal Electricity Storage Cost Arbitrage<p></p><div style="clear: both; text-align: justify;"><span style="text-align: left;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj_pEtUuWhqcwZWne74me50HXMAUoBlYjjbFTpHhZKnHR6J8h1ZaX0_jNo9GPT48x6TptHl9GBUcD9Bz0jmTTR9nbG-faTwH2uA91vZHe8BGVXpu5_nDOHTTiGaass8KuCan5GFW-AXSB6KOuJx0ckDjMOtp9aqc7iiwe998IvVlVtNI-DGjucLQeq9Q/s673/fig.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="623" data-original-width="673" height="296" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj_pEtUuWhqcwZWne74me50HXMAUoBlYjjbFTpHhZKnHR6J8h1ZaX0_jNo9GPT48x6TptHl9GBUcD9Bz0jmTTR9nbG-faTwH2uA91vZHe8BGVXpu5_nDOHTTiGaass8KuCan5GFW-AXSB6KOuJx0ckDjMOtp9aqc7iiwe998IvVlVtNI-DGjucLQeq9Q/s320/fig.png" width="320" /></a></div><div style="text-align: left;">As more and more solar and wind generation capacity is integrated into a power grid <a href="https://blog.kugelfish.com/2023/01/moving-variable-renewables-from-pay-as.html" target="_blank">without sufficient storage</a>, the intra-day electricity market prices tend to fluctuate between near-zero when there is an over-supply of renewable energy and quite expensive when fossil fuel power plants need to be brought online to cover the demand. The fewer hours these plants run and <a href="https://blog.kugelfish.com/2023/02/what-should-be-fair-cost-of-carbon.html" target="_blank">the more expensive the fossil fuel becomes</a> the more the range of this short term fluctuations will increase, unless they can potentially be substituted by a cheaper solution. </div><div style="text-align: left;"><span><br /></span></div><div style="text-align: left;"><span>Using electricity market data from the </span><a href="https://blog.kugelfish.com/2023/01/energy-data-analysis-with-pandas.html" target="_blank">ENTSO-E transparency platform</a><span>, we can determine what would be the maximum profit the operator of a hypothetical storage unit could make by deploying an optimal strategy of buying low and selling high.</span></div></span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;"><br /></span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;">The storage is assumed to have one unit of power, i.e. in this case it could either charge or discharge one MWh of energy in a given hour. The storage system can have a capacity of n hours and a round-trip efficiency of e, where half of which is assumed to be dissipated during charging and discharging respectively. We are also ignoring any effects of losses or congestion in the transmission network.</span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;"><br /></span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;">We are also assuming that the storage system is small enough to be able to follow along the market without moving it and it can use the perfect knowledge of the day-ahead forward market to choose the optimal charge and discharge points. While perfect market knowledge might not be realistic, renewable energy production in aggregate is relatively predictable - following roughly the accuracy of weather forecasting.</span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;"><br /></span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;">The following shows the maximum profit a storage system of capacity 1 (1MW/1MWh) and efficiency of 90% could have made in a few different European markets during the years 2019 and 2022 respectively:</span></div><div style="clear: both; text-align: justify;"><br /></div><div style="clear: both; text-align: justify;"><b>2019</b></div><div style="clear: both; text-align: justify;"><span style="text-align: left;">
<p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Country</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Profit</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Cycles</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Present value<br />(10 y, 5%)</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Germany</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€11,707.56</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">731</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€90,402.67</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Switzerland</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€6,672.77</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">610</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€51,525.36</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Spain</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€5,103.18</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">570</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€39,405.40</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">France</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€10,890.36</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">795</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€84,092.47</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Netherlands</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€10,438.41</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">733</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€80,602.64</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Denmark (East)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€9,032.15</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">626</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€69,743.87</td>
</tr>
</tbody>
</table><br /><b>2022</b><br /><p></p>
<p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Country</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Profit</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Cycles</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Present value <br />(10 y, 5%)</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Germany</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€75,791.35</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">729</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€585,240.71</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Switzerland</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€44,048.61</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">639</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€340,131.69</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Spain</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€36,046.15</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">701</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€278,338.82</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">France</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€68,775.37</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">782</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€531,065.18</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Netherlands</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€89,301.90</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">846</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€689,565.60</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Denmark (East)</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€79,355.55</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">749</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: medium; padding: 0px 20px 0px 0px; text-align: left; width: auto;">€612,762.52</td>
</tr>
</tbody>
</table><p></p>
</span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;">The year 2019 represents the last year of the cheap and abundant energy in Europe, not disrupted by pandemic or war, while 2022 was characterised by supply shortages and high fossil fuel prices. While in the years up to 2019 electricity would have been to cheap for any existing storage technology to be profitable for short-term storage arbitrage, the price landscape of 2022 would allow to quickly amortise even the still relatively expensive lithium-ion batteries, which today are mostly deployed at grid scale to provide more lucrative <a href="https://en.wikipedia.org/wiki/Ancillary_services_(electric_power)">ancilliary services</a>.</span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;"><br /></span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;">For the foreseeable future, times of cheap and abundant natural gas will not likely return in Europe, even if prices drop from the 2022 peaks. From this we can anticipate that electricity arbitrage could be a profitable use-case for a stand-alone battery electric storage systems in many European power grids - as long as this would be permitted by the local market regulation. Given that wind and solar producers will face increasing levels of curtailment of their potential production which cannot be absorbed by the transmission networks, storage will likely be co-located to form hybrid wind/solar + storage power plants that connect to the grid with a firmer delivery profile.</span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;"><br /></span></div><div style="clear: both; text-align: justify;"><span style="text-align: left;">In order to determine the optimal charge & discharge intervals in the market data time series, we use the dynamic programming optimisation algorithm below:</span></div><div style="clear: both; text-align: justify;"><br /></div><p></p>
<!--HTML generated using hilite.me--><div style="background: rgb(248, 248, 248); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #408080; font-style: italic;"># Stores the intermediate results of the optimisation process</span>
<span style="color: #408080; font-style: italic;">#</span>
<span style="color: #408080; font-style: italic;"># value: profit from completed trades so far, minus the cost base of current charge</span>
<span style="color: #408080; font-style: italic;"># cost: cost base for current charge in storage</span>
<span style="color: #408080; font-style: italic;"># count: number of charge or discharge events so far</span>
OptimisationState <span style="color: #666666;">=</span> namedtuple(<span style="color: #ba2121;">"OptimisationState"</span>, <span style="color: #ba2121;">"value cost count"</span>)
<span style="color: green; font-weight: bold;">def</span> <span style="color: blue;">max_value</span>(a, b):
<span style="color: green; font-weight: bold;">return</span> <span style="color: green;">max</span>(a,b, key<span style="color: #666666;">=</span> <span style="color: green; font-weight: bold;">lambda</span> item : item<span style="color: #666666;">.</span>value <span style="color: green; font-weight: bold;">if</span> item <span style="color: green; font-weight: bold;">else</span> <span style="color: #666666;">-</span>math<span style="color: #666666;">.</span>inf)
<span style="color: green; font-weight: bold;">def</span> <span style="color: blue;">charge</span>(state, rate, efficiency):
<span style="color: #408080; font-style: italic;"># Assuming half of the storage round-trip inefficiency is lost during charging,</span>
<span style="color: #408080; font-style: italic;"># leading to a higher cost</span>
charge_cost <span style="color: #666666;">=</span> rate <span style="color: #666666;">*</span> (<span style="color: #666666;">1</span> <span style="color: #666666;">+</span> (<span style="color: #666666;">1</span> <span style="color: #666666;">-</span> efficiency) <span style="color: #666666;">/</span> <span style="color: #666666;">2</span>)
<span style="color: green; font-weight: bold;">return</span> OptimisationState(state<span style="color: #666666;">.</span>value <span style="color: #666666;">-</span> charge_cost,
state<span style="color: #666666;">.</span>cost <span style="color: #666666;">+</span> charge_cost, state<span style="color: #666666;">.</span>count <span style="color: #666666;">+</span> <span style="color: #666666;">1</span>)
<span style="color: green; font-weight: bold;">def</span> <span style="color: blue;">discharge</span>(state, rate, efficiency, soc):
<span style="color: #408080; font-style: italic;"># Assuming half of the storage round-trip inefficiency is lost during discharge,</span>
<span style="color: #408080; font-style: italic;"># leading to a lower profit</span>
discharge_proceeds <span style="color: #666666;">=</span> rate <span style="color: #666666;">*</span> (<span style="color: #666666;">1</span> <span style="color: #666666;">-</span> (<span style="color: #666666;">1</span> <span style="color: #666666;">-</span> efficiency) <span style="color: #666666;">/</span> <span style="color: #666666;">2</span>)
<span style="color: #408080; font-style: italic;"># Use averaging for cost base, dividing total cost by the current state of charge</span>
<span style="color: green; font-weight: bold;">return</span> OptimisationState(state<span style="color: #666666;">.</span>value <span style="color: #666666;">+</span> discharge_proceeds,
state<span style="color: #666666;">.</span>cost <span style="color: #666666;">-</span> state<span style="color: #666666;">.</span>cost <span style="color: #666666;">/</span> soc, state<span style="color: #666666;">.</span>count <span style="color: #666666;">+</span> <span style="color: #666666;">1</span>)
<span style="color: green; font-weight: bold;">def</span> <span style="color: blue;">simulate_interval</span>(prev, rate, efficiency):
<span style="color: #ba2121; font-style: italic;">"""Simulate storage system optimisation for one step of the time series.</span>
<span style="color: #ba2121; font-style: italic;"> Storage can either hold, charge or discharge one unit of energy</span>
<span style="color: #ba2121; font-style: italic;"> during this interval.</span>
<span style="color: #ba2121; font-style: italic;"> Parameters:</span>
<span style="color: #ba2121; font-style: italic;"> prev: state of the optimisation before this interval</span>
<span style="color: #ba2121; font-style: italic;"> rate: unit cost of energy for this interval</span>
<span style="color: #ba2121; font-style: italic;"> efficiency: round-trip storage efficiency</span>
<span style="color: #ba2121; font-style: italic;"> Returns:</span>
<span style="color: #ba2121; font-style: italic;"> new state of the optimisation after this interval</span>
<span style="color: #ba2121; font-style: italic;"> """</span>
<span style="color: #408080; font-style: italic;"># simulate all holds (nothing changes)</span>
current <span style="color: #666666;">=</span> prev<span style="color: #666666;">.</span>copy()
<span style="color: #408080; font-style: italic;"># simulate all charges</span>
<span style="color: green; font-weight: bold;">for</span> idx, elem <span style="color: #aa22ff; font-weight: bold;">in</span> <span style="color: green;">enumerate</span>(prev[:<span style="color: #666666;">-1</span>]):
<span style="color: green; font-weight: bold;">if</span> elem:
current[idx <span style="color: #666666;">+</span> <span style="color: #666666;">1</span>] <span style="color: #666666;">=</span> max_value(current[idx <span style="color: #666666;">+</span> <span style="color: #666666;">1</span>],
charge(elem, rate, efficiency))
<span style="color: #408080; font-style: italic;"># simulate all discharges</span>
<span style="color: green; font-weight: bold;">for</span> idx, elem <span style="color: #aa22ff; font-weight: bold;">in</span> <span style="color: green;">enumerate</span>(prev[<span style="color: #666666;">1</span>:], start<span style="color: #666666;">=1</span>):
<span style="color: green; font-weight: bold;">if</span> elem:
current[idx <span style="color: #666666;">-</span> <span style="color: #666666;">1</span>] <span style="color: #666666;">=</span> max_value(current[idx <span style="color: #666666;">-</span> <span style="color: #666666;">1</span>],
discharge(elem, rate, efficiency, idx))
<span style="color: green; font-weight: bold;">return</span> current
<span style="color: green; font-weight: bold;">def</span> <span style="color: blue;">maximize_profit</span>(time_series, capacity, efficiency):
<span style="color: #ba2121; font-style: italic;">"""Maximize profit from energy arbitrage.</span>
<span style="color: #ba2121; font-style: italic;"> Using an idealised storage model and perfect knowledge of the market.</span>
<span style="color: #ba2121; font-style: italic;"> Parameters:</span>
<span style="color: #ba2121; font-style: italic;"> time_series: Pandas time-series of day-ahead electricity market prices.</span>
<span style="color: #ba2121; font-style: italic;"> capacity: Storage capacity in multiples of the energy unit of the time series.</span>
<span style="color: #ba2121; font-style: italic;"> efficiency: Round-trip storage efficiency</span>
<span style="color: #ba2121; font-style: italic;"> Returns:</span>
<span style="color: #ba2121; font-style: italic;"> Tuple of profit and number of charge/discharge cycles,</span>
<span style="color: #ba2121; font-style: italic;"> resulting from optimal charge/discharge pattern</span>
<span style="color: #ba2121; font-style: italic;"> """</span>
<span style="color: #408080; font-style: italic;"># initialise empty system state</span>
<span style="color: #408080; font-style: italic;"># (One OptimisationState record per possible state-of-charge value)</span>
state <span style="color: #666666;">=</span> [<span style="color: green;">None</span>] <span style="color: #666666;">*</span> (capacity <span style="color: #666666;">+</span> <span style="color: #666666;">1</span>)
state[<span style="color: #666666;">0</span>] <span style="color: #666666;">=</span> OptimisationState(<span style="color: #666666;">0</span>, <span style="color: #666666;">0</span>, <span style="color: #666666;">0</span>)
<span style="color: green; font-weight: bold;">for</span> index, value <span style="color: #aa22ff; font-weight: bold;">in</span> time_series<span style="color: #666666;">.</span>items():
state <span style="color: #666666;">=</span> simulate_interval(state, value, efficiency)
<span style="color: #408080; font-style: italic;">#print (index, value, state)</span>
<span style="color: green; font-weight: bold;">return</span> (state[<span style="color: #666666;">0</span>]<span style="color: #666666;">.</span>value, state[<span style="color: #666666;">0</span>]<span style="color: #666666;">.</span>count<span style="color: #666666;">/2</span>)
</pre></div>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-79119427510153609152023-05-21T00:40:00.006+02:002023-07-05T12:46:46.414+02:00It is 12 noon - do you know where your car is?<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi36wN4R1qUtl1zXb2vV9SJSAtLg0jVC6b9NawqGW4t4s_hrfYs1vLBk6-e9ud5V7T5Bo_6O38oqzoq823CJRNky1U8IEzJVlDwZD_Eve4plVE6Ud0iJn5qGNkTuTiPOqL5F668KTiJc2_62fW8Z6e4Mv8bSEXm9TyULjTdIUCx-xFyPJDYFL-ymM9Jsg/s1024/IMG_20180714_114900.jpg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="798" data-original-width="1024" height="297" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi36wN4R1qUtl1zXb2vV9SJSAtLg0jVC6b9NawqGW4t4s_hrfYs1vLBk6-e9ud5V7T5Bo_6O38oqzoq823CJRNky1U8IEzJVlDwZD_Eve4plVE6Ud0iJn5qGNkTuTiPOqL5F668KTiJc2_62fW8Z6e4Mv8bSEXm9TyULjTdIUCx-xFyPJDYFL-ymM9Jsg/w382-h297/IMG_20180714_114900.jpg" width="382" /></a></div>For most privately owned cars, the most common answers are: at home, at work or somewhere on the road.<p></p><p>But why does this matter?</p><p>The increased installation of solar panels is leading to a more pronounced "<a href="https://en.wikipedia.org/wiki/Duck_curve" target="_blank">duck curve</a>" in many power grids - with an over-supply of electricity around noon on sunny days and demand ramping up rapidly in the evening as the sun sets and people are returning home.</p><p><a href="https://blog.kugelfish.com/2023/01/moving-variable-renewables-from-pay-as.html" target="_blank">Large amounts of energy storage</a> will be needed to "flatten the duck". While VRES like solar and wind have generally become the cheapest sources of electricity production, storage is still very expensive.</p><p>At the same time, we are seeing a strong trend to replace traditional cars with battery electric vehicles (BEV). For better or for worse, BEV are essentially large batteries on wheels. Because of this battery, BEV are significantly more expensive to manufacture and have a roughly 30% higher initial "carbon debt" than comparable ICE cars. On the other hand, once purchased, their carbon footprint improves the more they are (legitimately) used.</p><p>According to the EV database, the <a href="https://ev-database.org/cheatsheet/useable-battery-capacity-electric-car" target="_blank">average BEV battery size</a> is about 70kWh and the <a href="https://ev-database.org/cheatsheet/energy-consumption-electric-car" target="_blank">average energy consumption</a> is 19kW per 100km. The average daily distance driven per car is about 30-40km (EU - US), which means that the average BEV needs about 6-8 kWh of energy per day for its driving needs and that a battery charge can last on average 8-10 days.</p><p>In order for electric cars to help "flatten the duck curve", charging stations would need to be available where the cars tend to be in the middle of the day - which for many commuters would be on corporate parking lot. Taking advantage of excess energy at mid-day on a sunny day will also require a smart charging infrastructure which delays the charging to the right time based on expected electricity supply as well as knowledge of the car's expected driving schedule.</p><p>Average daily electricity consumption (without heat and hot water) for a single person household in Switzerland is <a href="https://pubdb.bfe.admin.ch/de/publication/download/10559" target="_blank">estimated</a> around 6kWh - mostly for cooking and refrigerating food as well as washing and drying laundry. Assuming a 10 minute shower per day, we can add an additional 6kWh for hot water. Heating is harder to estimate, depending on the age and state of building as well as the climate and season.</p><p>Which means that the average BEV could support the average total electricity needs of its owner for 3-4 days without recharging, making it a significant source of potential long-duration storage.</p><p>For this use-case to work, cars would need to be equipped with bi-directional chargers (currently only in a few models) as well as a bi-directional charging connections and energy management systems at home or wherever the store electricity should be consumed.</p><p>Energy economics alone will unlikely justify the necessary investments. However, we also generally do not make automative purchase decisions purely on economic reasons - otherwise many of us would not own a car or at least not as large a car as we do. Beyond the utilitarian purpose of mobility, cars often represent intangible "value beyond" like fun, freedom, independence, rugged individualism, identity or security.</p><p>The idea of increased autarky and energy independence from the grid might well align with the mindset of many owners of large premium BEVs. As severe weather becomes more common across the globe and the power grid might become less reliable, being able to use the car as backup power for the whole house could provide a sense of security for which many owners are willing to pay a premium.</p><p>A particularly fitting symbol for this brand of rugged individualism might be the electric Ford F150 lightning pickup truck with a 130 kWh battery. Which is more than large enough to power even a large house for days if not weeks.</p><p>For private car owners, V2G (vehicle to grid) <a href="https://smartsolarcharging.eu/wp-content/uploads/sites/274/2017/08/A.-Kaufmann-Master_Thesis_V2G_BusinessModel.pdf" target="_blank">might never be a particularly appealing use-case</a>, while home backup power and increased autonomy (V2H: vehicle to home) might be more in line with the mindset of private car owners around freedom or independence. If car batteries are being used to supply electricity, most of the action might be behind the meter (BTM).</p><p>If a car is typically not at home in the garage during the mid-day solar peak, then it cannot the store excess production from the rooftop solar panels. However it could store the excess power from solar panels at a corporate office parking lot, taking the energy home to be used in the evening and over night.</p><p>If BEV are to become the central hub of a BYOE ("Bring Your Own Electricity") solution, they need to become much better at predicting their user's mobility needs, as well as the energy supply and demand profiles for each of the locations where the car is regularly connected to a grid.</p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-34078597362363394562023-02-25T02:37:00.010+01:002023-02-25T16:17:56.893+01:00What should be the fair cost of Carbon?<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWYlU0bgt9gPysNvg6FXtTKOSWr7FSvZT7r3NPDD8I582NUd2krjHL6o2BdOxqvWqVs2zgaZNPxxlEIkL_fhDFDmmna5kjcZeOpm81NAjNJTDG_xilrdXVbdRxpVxbBDduFUSUKMPmJiYP3_M1DqHXqvK0MoXqY1T9bD9Z3NtrSVACjxCdDndvNVZoWw/s899/biker_santa.jpg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="752" data-original-width="899" height="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWYlU0bgt9gPysNvg6FXtTKOSWr7FSvZT7r3NPDD8I582NUd2krjHL6o2BdOxqvWqVs2zgaZNPxxlEIkL_fhDFDmmna5kjcZeOpm81NAjNJTDG_xilrdXVbdRxpVxbBDduFUSUKMPmJiYP3_M1DqHXqvK0MoXqY1T9bD9Z3NtrSVACjxCdDndvNVZoWw/w365-h306/biker_santa.jpg" width="365" /></a></div>With an energy crisis looming across Europe, some people were joking that those who follow the American Santa Claus traditions should rather be naughty than nice this year, because coal would be more valuable this winter than toys.<p></p><p>Also this winter, the <a href="https://www.ft.com/content/7a0dd553-fa5b-4a58-81d1-e500f8ce3d2a" target="_blank">price per ton of carbon emissions exceeded €100</a> in the EU emission trading system for the first time - presumably triggered by a shift in electricity generation from gas to coal, which is more carbon intensive per MWh of electricity generated.</p><p>Oil and gas prices have surged last year far beyond the levels which would have been for example the CO2 tax on fossil fuels rejected by Swiss voters in 2021. This might have provided an unintended stress test of what the social and economic impact would be if we were serious about decarbonisation and has spurred a level of attention and investment into "alternative energy" (now called renewable or green energy) not seen since the oil crises in the 1970ies.</p><p>In order to keep the momentum of decarbonisation, the price for CO2 (and equivalent GHG) emissions likely needs to continue to remain much higher than before 2021, but ideally without the windfall flowing into the pockets of autocrats and oil companies.</p><p>But without a global supply crisis, how should policy makers set the price for carbon emissions in a principled way?</p><p>There are several approaches or mental models that could be used to determine what should be the "right" cost for CO2 emissions.</p><p>In classical economic thinking, CO2 emissions are an externality that could be priced in for example by imposing a <a href="https://en.wikipedia.org/wiki/Pigouvian_tax" target="_blank">Pigouvian tax</a>. Most carbon taxes or emission trading schemes like the aforementioned EU ETS seem to operate at least initially on this principle.</p><p>There have been scientific attempts to determine the externality cost or <a href="https://en.wikipedia.org/wiki/Social_cost_of_carbon" target="_blank">social cost of CO2 emissions</a>. These estimates only work over a somewhat stable range of assumptions and break down completely under asymptotic conditions: after all, what would be the cost of the end of civilisation as we know it that would even mean the end of concepts like "economics" or "cost"? Even with more bounded assumptions, estimates range from tens to thousands of dollars per incremental ton of CO2 emissions depending of the source - <a href="https://www.nature.com/articles/s41586-022-05224-9" target="_blank">trending upwards as the models are updated every few years</a>. </p><p>Another approach would be to treat carbon emissions like other harmful emissions that need to be regulated and ultimately banned. This approach has worked well to combat air pollution and acid rain (at least in industrialised countries) and the degradation of the ozone layer globally.</p><p>This approach would be well aligned with the current thinking about "net-zero" goals which is equivalent to ban all net CO2 emissions from human activities by a certain date. At this point, the cost of further fossil CO2 emissions would then be equal to the cost of "cleanup" in the form of removal and long-term storage of the same amount of CO2. Stable and long-term removal of CO2 from the atmosphere is currently <a href="https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE4QO0D" target="_blank">very limited in capacity</a> and costs between hundreds to thousands of dollars per ton.</p><p>There is a larger market for voluntary compensations that are much cheaper, but largely not ensured to last over geological time periods and are prone over-statement, fraud or even to mafia style protection rackets like: "<a href="https://www.zeit.de/wirtschaft/2023-01/co2-certificates-fraud-emissions-trading-climate-protection-english/komplettansicht" target="_blank">I have a pretty little forrest here - wouldn't it be a shame if something were to happen to it...</a>".</p><p>Using long-term, geologically stable carbon removal as a pricing benchmark for CO2 emissions might make sense, but might also mis-direct resources into scaling negative emission processes beyond what makes sense in the long term.</p><p>We could also consider the price of CO2 emissions to be a <a href="https://en.wikipedia.org/wiki/Steering_tax" target="_blank">steering or incentive tax</a> with the purpose of achieving a goal of reducing emissions to net zero over some time period. Once the goals are defined, carbon prices could be dynamically adjusted based on how well certain reduction targets are reached or not, similar how central banks set interest rates to achieve certain price stability or employment objectives. Many of the carbon trading systems around the world have introduced <a href="https://climate.ec.europa.eu/system/files/2020-06/study_market_stability_measures_en.pdf" target="_blank">market stability measures</a>, for example the Market Stability Reserve (MSR) of the EU ETS. While these measures might not be formally linked to effectiveness, they could certainly evolve in this direction. But as central banks around the world have political independence to shield them from being ousted after taking unpopular measures, any governing body with carbon pricing power would likely require a similar level of political protection.</p><p>It is not clear how high the price for CO2 emissions would have to be under this model, but given the current state of progress against stated goals of decarbonisation, it would likely need to be significantly higher than today.</p><p><br /></p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-38356571700890710242023-02-10T14:19:00.005+01:002023-10-29T00:58:43.242+02:00From MrBeast to MrBot?<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-BEjWXomACDVqXa-ClGrnV0cFh8Kn37l8JwY6soHgc_klp2s_qcgsSJveVcVk6b8ZGs3sCzoRanuiNCZZAy-p8Eea3PbTAxUKPGwi9r18ZSTtd1JzJ8_-fj09l4bPvMRgknYJn-tXG831f-Gy2G7ripn2CNFilzuegwj_UwewghKCK848OSb3pDzOPw/s1784/mrbot.jpeg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1784" data-original-width="1021" height="401" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-BEjWXomACDVqXa-ClGrnV0cFh8Kn37l8JwY6soHgc_klp2s_qcgsSJveVcVk6b8ZGs3sCzoRanuiNCZZAy-p8Eea3PbTAxUKPGwi9r18ZSTtd1JzJ8_-fj09l4bPvMRgknYJn-tXG831f-Gy2G7ripn2CNFilzuegwj_UwewghKCK848OSb3pDzOPw/w230-h401/mrbot.jpeg" width="230" /></a></div>Generative AI seems to have entered the mainstream hype cycle with the recent release of <a href="https://chat.openai.com/chat" target="_blank">ChatGPT</a> becoming the fastest growing website to reach more than 100 million users, generating a lot of speculation about AI disrupting and redefining how we navigate and access knowledge and information.<p></p><p>Having worked with but not on machine learning the last few years, I am myself amused, amazed and somewhat terrified by the impressive progress in what publicly available text and image generators are able to do now.</p><p>At its current state, ChatGPT is essentially an entertainment platform in the form of an infuriatingly polite, well-read, hallucinating bullshitter. One should not ask it any question for which we don't already know the answer, but like a mechanical automation in a victorian era parlour, we can marvel at how well it performs some amazing tricks some times or laugh at how it spectacularly fails some other times.</p><p>Unless there is an unexpected ceiling, another decades long "<a href="https://en.wikipedia.org/wiki/AI_winter" target="_blank">AI winter</a>", we can expect generative AI to be some of the most significant disruption of our lives since the introduction of computers, the Internet or mobile phones.</p><p>Most of the large AI research labs are working frantically at trying to teach the latest generation of large language models to be less fast and loose with the facts and become more like a responsible librarian or researcher than a fast talking con artist. If or when they succeed, this will have tremendous impact on the life of any knowledge worker. Anybody who reads and writes for a living, who analyses sources and compiles reports based on them. In short, almost anybody working in an office today.</p><p>But there are also fields where truth and facts are far less important than sheer popularity: for example in entertainment and politics. In the realms of commercial pop-culture, success is very easy to measure, in terms of clicks for web publishing, best-seller lists for books, billboard 100 charts for music or Nielsen ratings on TV or views, likes, fans or subscriber counts on streaming video or social media distribution platforms. And one thing that machine learning is good at, is to optimise for a well defined success metric.</p><p>Over the last few decades, the digital revolution and the internet has reduced the cost of content creation and distribution to almost zero. Making a record or a TV show no longer requires a studio with equipment worth millions of dollars but can be done today with a cellphone or a laptop. Ironically this democratisation of access to the means of content creation has fuelled a winner-takes-it all race, where fewer and fewer of the most famous and successful creators manage to stand out of the noise and get the lion's share of attention, fame and fortune. And for the last decade, machine learning driven recommendation systems of content distribution platforms have indirectly played a role as king-maker and gatekeeper.</p><p>Would it be so hard to imagine how only slight evolutions of the current generative AI models could flood the current content distribution channels and optimise for what we want to see, hear and read in a similar but more systematic way the successful commercial content creators have been doing for decades?</p><p>AI creators might not (yet) sweep the Nobel prize for literature, the Pulitzer prize for journalism or the top outstanding achievement awards for music film or TV, but that's not really where they money is. Would it really be so hard to imagine an AI generated paperback novels ghost-written celebrity memoirs, clickbait articles, pop music tracks or social media influencer post?</p><p><br /></p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-22234597786635208592023-01-25T22:24:00.021+01:002023-05-31T13:16:38.429+02:00Moving variable renewables from "pay-as-produced" to pay-as-needed"<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiK8537ydOUXBO-B_yQsj9N0tPrtGlDz-y65z2vH0jHc2qFwkDPOhDbNdg0AhZW1nMa77ljqLNCTxFTZhCNnIdZMUA7JmKlMIrTNa06xbb4BvLTL8l71TgfDCbQ8I1XXqqt3PHbAHc0rrx21y9S3aFBksEAGPdR65vOhhZPOOooqK-XjHmpkkcbefLBvg/s629/fig.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="447" data-original-width="629" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiK8537ydOUXBO-B_yQsj9N0tPrtGlDz-y65z2vH0jHc2qFwkDPOhDbNdg0AhZW1nMa77ljqLNCTxFTZhCNnIdZMUA7JmKlMIrTNa06xbb4BvLTL8l71TgfDCbQ8I1XXqqt3PHbAHc0rrx21y9S3aFBksEAGPdR65vOhhZPOOooqK-XjHmpkkcbefLBvg/w397-h282/fig.png" width="397" /></a></div>In some of the major European power grids, the penetration of <a href="https://en.wikipedia.org/wiki/Variable_renewable_energy" target="_blank">variable renewable energy sources</a> (VRES) has reached a level where their variability can no longer be easily absorbed by the grid as minor noise in the daily fluctuations of supply and demand.<p></p><p>Taking for example the German grid (~ 60 GW load), the spot market prices now routinely "flat-line" at or below zero during nights and week-end on particularly blustery days. This <a href="https://www.tagesschau.de/wirtschaft/windkraft-probleme-101.html" target="_blank">forces wind farms to shut down </a>(curtailment) during times when they would be at their most productive. Because the production potential of wind & solar is each highly synchronised across a relatively large geographic area, this trend will only increase as more wind and solar capacity is added to the same grid and profitably selling renewable energy "as-produced" will become increasingly difficulty.</p><p>For the next stages of integration into the grid, wind & solar resources might have to be combined with storage, moving away from delivering power whenever available and moving towards delivering whenever it is needed. An alternative strategy might be to deploy new resources in a more "contrarian" mode - for example <a href="https://www.zhaw.ch/de/lsfm/institute-zentren/iunr/oekotechnologien-energiesysteme/erneuerbare-energien/solarenergie/alpenstrom-davos/" target="_blank">high-alpine</a> or <a href="https://next2sun.com/en/" target="_blank">vertical east-west oriented bi-facial </a>solar plants that don't follow the summer mid-day peak.</p><p>Using the granular time-series data from the <a href="https://transparency.entsoe.eu/" target="_blank">ENTSO-E open-data platform</a> presented in the <a href="https://blog.kugelfish.com/2023/01/energy-data-analysis-with-pandas.html" target="_blank">previous post</a>, we can look at some of the pattern of wind and solar availability.</p><p>The following charts show the solar and on-shore wind forecasts for Germany, aggregated by hour of day over a 3 month period of 2002 for each of the 4 seasons:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgILWzAxUlnqXDDpXkaLZriMluK0FShxfOSa1sLU6E2SQ0esxDsAWJY8c56tS2rdlVf3uxEP747SruA53_X4dNFBRxbOle6b9Z9yZXeGsVNeIq-qyvputBfsTITSV9chcVSLMkYIwVvY6sYHQ_oiij23b49CmoJeJBX90Wv8SDB0CQ2wp2o_EDxNPSPQA/s1345/Germany.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1264" data-original-width="1345" height="602" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgILWzAxUlnqXDDpXkaLZriMluK0FShxfOSa1sLU6E2SQ0esxDsAWJY8c56tS2rdlVf3uxEP747SruA53_X4dNFBRxbOle6b9Z9yZXeGsVNeIq-qyvputBfsTITSV9chcVSLMkYIwVvY6sYHQ_oiij23b49CmoJeJBX90Wv8SDB0CQ2wp2o_EDxNPSPQA/w640-h602/Germany.png" width="640" /></a></div><p>We can see that for this particular part of northern, continental Europe, wind is generally stronger in the winter half of the year and due to the high latitude, solar is much stronger in the summer than in the winter. Wind also shows a much higher day-to-day variance than solar, but practically evens out in terms of average intra-day patterns, while solar energy product is concentrated more deterministically on a few hours around mid-day. When combining the two types of resources, we (unsurprisingly) see a lower volatility than for each of the two sources. Before starting the analysis, I would have expected a more pronounced pattern of wind distribution across hours of the day (e.g. more wind during the night, less during mid-day) as well as possibly stronger negative correlation (less wind on blue-sky & sunny days and vice versa.).</p><p>Another way to look at the mismatch between supply and demand is to determine what amount of storage would be needed to bridge the gap:</p><p></p>
<p></p><table class="dataframe">
<thead>
<tr style="text-align: right;">
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Type</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Avg Power (GW)</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Total Energy (GWh)</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Storage Capacity (GWh)</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">St. Power (GW)</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">St. Duration (h)</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Capcity Ratio (%)</th>
<th style="background-color: white; border-bottom: 2px solid rgb(48, 84, 150); color: #305496; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">Stored Energy Ratio (%)</th>
</tr>
</thead>
<tbody>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">solar as base load</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">6.4</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">55771.0</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">14156.6</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">31.7</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">447</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">25.38</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">60.87</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">wind as base load</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11.3</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99352.3</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18382.9</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">29.7</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">620</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">18.5</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">33.82</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">combined as base load</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">17.7</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">155123.2</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11826.1</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">42.1</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">281</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">7.62</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">28.0</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">solar as 11.4% of load</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">6.4</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">55771.0</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">15077.0</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">31.5</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">479</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">27.03</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">58.71</td>
</tr>
<tr>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">wind as 20.3% of load</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">11.3</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">99352.3</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">15680.7</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">29.4</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">533</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">15.78</td>
<td style="background-color: #d9e1f2; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">32.76</td>
</tr>
<tr>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">combined as 31.7% of load</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">17.7</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">155123.2</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">9237.4</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">37.8</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">244</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">5.95</td>
<td style="background-color: white; color: black; font-family: "Century Gothic", sans-serif; font-size: small; padding: 0px 20px 0px 0px; text-align: left; width: auto;">25.65</td>
</tr>
</tbody>
</table><p></p>
<p></p><p>This <a href="https://blog.kugelfish.com/2023/01/energy-data-analysis-with-pandas.html" target="_blank">ludicrously over-simplistic simulation</a> assumes an idealised storage of 100% efficiency that would redistribute the variable rate wind & solar production into either a constant output corresponding to the yearly average ("base-load") or to a constant fraction of the actual load curve that corresponds to the yearly share of energy production. See for example <a href="https://www.cell.com/joule/fulltext/S2542-4351(19)30300-9" target="_blank">this paper</a> for a more realistic/sophisticated approach to storage impact simulation.</p><p>We can see that in general solar requires a higher rate of storage capacity than wind - about 28% of storage capacity per amount of energy produced and about 60% of the energy needs to be cycled through storage instead of used directly when produced.</p><p>Combining both sources significantly reduced the storage capacity needed, to about 6% of the energy produced and about 25% of the energy requiring storage.</p><p>This is still a daunting amount - equivalent to about 450 times the capacity of the new <a href="https://en.wikipedia.org/wiki/Nant_de_Drance_Hydropower_Plant" target="_blank">Nant de Drance</a> pumped hydro power plant in Switzerland or the battery packs of 90M Tesla model S. Also the cycle efficiency of real storage systems are far from 100%. They range from 90% for batteries to about 70-80% for pumped hydro to less than 50% for P-to-X.</p><p>As we move into the next phase of large scale renewable energy deployment, storage technology will be the space to watch.</p><p><br /></p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-39145996354347254752023-01-22T20:13:00.097+01:002023-01-26T23:19:46.094+01:00Energy Data Analysis with Pandas<p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2TBhzTwaAT2UMVjd01YKrM93ZlVgiiPrCjAbfkZcSehhpwOBt0vww0k3g21yH0j-br5Z4vOw4pEiSLizjqpeEsXn7QTNuT5qypkpZcO2OF34wSE4JlWsQnRGQEFzJ8H0ndY5v8HSLhdyuS_WlZ_JbNEWERNX8cVc0rvIm7_WX84T7WuB-7kEe89JbLQ/s2318/IMG_4249.JPG" style="clear: left; display: inline; float: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="2318" data-original-width="1799" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2TBhzTwaAT2UMVjd01YKrM93ZlVgiiPrCjAbfkZcSehhpwOBt0vww0k3g21yH0j-br5Z4vOw4pEiSLizjqpeEsXn7QTNuT5qypkpZcO2OF34wSE4JlWsQnRGQEFzJ8H0ndY5v8HSLhdyuS_WlZ_JbNEWERNX8cVc0rvIm7_WX84T7WuB-7kEe89JbLQ/w310-h400/IMG_4249.JPG" width="310" /></a>The organisation of Europe's power grid operators (<a href="https://en.wikipedia.org/wiki/European_Network_of_Transmission_System_Operators_for_Electricity" target="_blank">ENTSO-E</a>) is providing an open-data <a href="https://transparency.entsoe.eu/dashboard/show" target="_blank">transparency platform</a> with a lot of interesting data about the state of the power grid in its various member countries. This data is among others also used to power websites like <a href="https://app.electricitymaps.com/map">https://app.electricitymaps.com/map</a>.</p><p>In order to access the <a href="https://transparency.entsoe.eu/content/static_content/Static%20content/web%20api/Guide.html" target="_blank">REST API,</a> one needs to register a user account on the site and request API access via an email to the support help-desk following the instructions <a href="https://transparency.entsoe.eu/content/static_content/Static%20content/web%20api/Guide.html#_query_via_web_browser" target="_blank">here</a>.</p><p>There is also a <a href="https://github.com/EnergieID/entsoe-py" target="_blank">Python client </a>for this API which also converts the raw XML data into Pandas dataframes. <a href="https://pandas.pydata.org/docs/index.html" target="_blank">Pandas</a> is a Swiss army knives for dataset manipulations and one of the reasons why Python is so popular among data scientists.</p><p>The following code shows how to do simple ad-hoc analysis with the granular time-series data returned by the API. The example shows a very over-simplistic back of the envelope estimate of the idealized storage that would be needed to align the variable solar and wind energy production with the fluctuation of demand over the same time that is used in the following blog post on <a href="https://blog.kugelfish.com/2023/01/moving-variable-renewables-from-pay-as.html" target="_blank"><i>Moving variable renewables from "pay-as-produced" to pay-as-needed"</i></a></p><p>
<!--HTML generated using hilite.me--></p><div style="background: rgb(248, 248, 248); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: green; font-weight: bold;">from</span> <span style="color: blue; font-weight: bold;">entsoe</span> <span style="color: green; font-weight: bold;">import</span> EntsoePandasClient
<span style="color: green; font-weight: bold;">import</span> <span style="color: blue; font-weight: bold;">pandas</span> <span style="color: green; font-weight: bold;">as</span> <span style="color: blue; font-weight: bold;">pd</span>
client <span style="color: #666666;">=</span> EntsoePandasClient(api_key<span style="color: #666666;">=</span><span style="color: #ba2121;">'<api key>'</span>)
start <span style="color: #666666;">=</span> pd<span style="color: #666666;">.</span>Timestamp(<span style="color: #ba2121;">'20220101'</span>, tz<span style="color: #666666;">=</span><span style="color: #ba2121;">'Europe/Madrid'</span>)
end <span style="color: #666666;">=</span> pd<span style="color: #666666;">.</span>Timestamp(<span style="color: #ba2121;">'20230101'</span>, tz<span style="color: #666666;">=</span><span style="color: #ba2121;">'Europe/Madrid'</span>)
country <span style="color: #666666;">=</span> <span style="color: #ba2121;">'ES'</span>
country_name <span style="color: #666666;">=</span> <span style="color: #ba2121;">'Spain'</span>
<span style="color: #408080; font-style: italic;"># query day-ahead wind & solar forecast & actual load timeseries</span>
forecast <span style="color: #666666;">=</span> client<span style="color: #666666;">.</span>query_wind_and_solar_forecast(country, start<span style="color: #666666;">=</span>start, end<span style="color: #666666;">=</span>end)
load <span style="color: #666666;">=</span> client<span style="color: #666666;">.</span>query_load(country, start<span style="color: #666666;">=</span>start, end<span style="color: #666666;">=</span>end)
<span style="color: #408080; font-style: italic;"># Resample time-series into uniform hourly resolution for easy</span>
<span style="color: #408080; font-style: italic;"># "integration" from power to energy (MH / MWh)</span>
forecast <span style="color: #666666;">=</span> forecast<span style="color: #666666;">.</span>resample(<span style="color: #ba2121;">'H'</span>)<span style="color: #666666;">.</span>agg(<span style="color: #ba2121;">'mean'</span>)
load <span style="color: #666666;">=</span> load<span style="color: #666666;">.</span>resample(<span style="color: #ba2121;">'H'</span>)<span style="color: #666666;">.</span>agg(<span style="color: #ba2121;">'mean'</span>)
forecast[<span style="color: #ba2121;">'combined'</span>] <span style="color: #666666;">=</span> forecast[<span style="color: #ba2121;">'Solar'</span>]<span style="color: #666666;">.</span>add(forecast[<span style="color: #ba2121;">'Wind Onshore'</span>])
<span style="color: #408080; font-style: italic;"># Averages off all the metrics over the</span>
load_mean <span style="color: #666666;">=</span> load[<span style="color: #ba2121;">'Actual Load'</span>]<span style="color: #666666;">.</span>mean()
solar_mean <span style="color: #666666;">=</span> forecast[<span style="color: #ba2121;">'Solar'</span>]<span style="color: #666666;">.</span>mean()
wind_mean <span style="color: #666666;">=</span> forecast[<span style="color: #ba2121;">'Wind Onshore'</span>]<span style="color: #666666;">.</span>mean()
combined_mean <span style="color: #666666;">=</span> forecast[<span style="color: #ba2121;">'combined'</span>]<span style="color: #666666;">.</span>mean()
<span style="color: #408080; font-style: italic;"># "Simulate" storage to reshape generation into desired output</span>
sim <span style="color: #666666;">=</span> pd<span style="color: #666666;">.</span>DataFrame()
<span style="color: #408080; font-style: italic;"># Instant generation power is set to hourly renewable forecast</span>
sim[<span style="color: #ba2121;">'gen_power'</span>] <span style="color: #666666;">=</span> forecast[<span style="color: #ba2121;">'combined'</span>]
<span style="color: #408080; font-style: italic;"># Set target output to a constant fraction of the actual load that corresponds</span>
<span style="color: #408080; font-style: italic;"># to the yearly share of energy production potential</span>
sim[<span style="color: #ba2121;">'out_power'</span>] <span style="color: #666666;">=</span> load[<span style="color: #ba2121;">'Actual Load'</span>] <span style="color: #666666;">*</span> (combined_mean <span style="color: #666666;">/</span> load_mean)
<span style="color: #408080; font-style: italic;"># Compute surplus/shortfall power for each hour as the flow in and out of storage</span>
sim[<span style="color: #ba2121;">'charge_discharge_power'</span>] <span style="color: #666666;">=</span> sim[<span style="color: #ba2121;">'gen_power'</span>]<span style="color: #666666;">.</span>sub(sim[<span style="color: #ba2121;">'out_power'</span>])
<span style="color: #408080; font-style: italic;"># Storage 'state of charge" as the running tab of charge & discharge contributions</span>
sim[<span style="color: #ba2121;">'storage_soc'</span>] <span style="color: #666666;">=</span> sim[<span style="color: #ba2121;">'charge_discharge_power'</span>]<span style="color: #666666;">.</span>cumsum()
total_energy <span style="color: #666666;">=</span> sim[<span style="color: #ba2121;">'out_power'</span>]<span style="color: #666666;">.</span>sum()
<span style="color: #408080; font-style: italic;"># Storage connection power needed is highes charge</span>
<span style="color: #408080; font-style: italic;"># or discharge movement for any of the time intervals</span>
storage_power <span style="color: #666666;">=</span> <span style="color: green;">max</span>(<span style="color: green;">abs</span>(sim[<span style="color: #ba2121;">'charge_discharge_power'</span>]<span style="color: #666666;">.</span>max()),
<span style="color: green;">abs</span>(sim[<span style="color: #ba2121;">'charge_discharge_power'</span>]<span style="color: #666666;">.</span>min()))
<span style="color: #408080; font-style: italic;"># Storage capacity is the delta between highest and lowest</span>
<span style="color: #408080; font-style: italic;"># fill-levels of storage state of charge</span>
storage_capacity <span style="color: #666666;">=</span> sim[<span style="color: #ba2121;">'storage_soc'</span>]<span style="color: #666666;">.</span>max() <span style="color: #666666;">-</span> sim[<span style="color: #ba2121;">'storage_soc'</span>]<span style="color: #666666;">.</span>min()
<span style="color: #408080; font-style: italic;"># Storage energy production is "integral" over outflows only</span>
stored_energy <span style="color: #666666;">=</span> sim[sim[<span style="color: #ba2121;">'charge_discharge_power'</span>] <span style="color: #666666;">></span> <span style="color: #666666;">0</span>][<span style="color: #ba2121;">'charge_discharge_power'</span>]<span style="color: #666666;">.</span>sum()
storage_duration <span style="color: #666666;">=</span> storage_capacity <span style="color: #666666;">/</span> storage_power
storage_ratio <span style="color: #666666;">=</span> storage_capacity <span style="color: #666666;">/</span> total_energy
stored_energy_ratio <span style="color: #666666;">=</span> stored_energy <span style="color: #666666;">/</span> total_energy
<span style="color: green; font-weight: bold;">print</span> (<span style="color: #ba2121;">'VRES generation potential for </span><span style="color: #bb6688; font-weight: bold;">%s</span><span style="color: #ba2121;">: </span><span style="color: #bb6688; font-weight: bold;">%.2f%%</span><span style="color: #ba2121;"> of load (</span><span style="color: #bb6688; font-weight: bold;">%.2f%%</span><span style="color: #ba2121;"> Solar / </span><span style="color: #bb6688; font-weight: bold;">%.2f%%</span><span style="color: #ba2121;"> Wind)'</span>
<span style="color: #666666;">%</span> (country_name, combined_mean <span style="color: #666666;">/</span> load_mean <span style="color: #666666;">*</span> <span style="color: #666666;">100</span>,
solar_mean <span style="color: #666666;">/</span> load_mean <span style="color: #666666;">*</span> <span style="color: #666666;">100</span>, wind_mean <span style="color: #666666;">/</span> load_mean <span style="color: #666666;">*</span> <span style="color: #666666;">100</span>))
<span style="color: green; font-weight: bold;">print</span> (<span style="color: #ba2121;">'Energy output </span><span style="color: #bb6688; font-weight: bold;">%.3f</span><span style="color: #ba2121;"> TWh'</span> <span style="color: #666666;">%</span> (total_energy <span style="color: #666666;">/</span> <span style="color: #666666;">1000000</span>,))
<span style="color: green; font-weight: bold;">print</span> (<span style="color: #ba2121;">'Storage dimensions: </span><span style="color: #bb6688; font-weight: bold;">%.3f</span><span style="color: #ba2121;"> TWh @ </span><span style="color: #bb6688; font-weight: bold;">%.3f</span><span style="color: #ba2121;"> GW (</span><span style="color: #bb6688; font-weight: bold;">%d</span><span style="color: #ba2121;"> h duration)'</span>
<span style="color: #666666;">%</span> (storage_capacity <span style="color: #666666;">/</span> <span style="color: #666666;">1000000</span>, storage_power <span style="color: #666666;">/</span> <span style="color: #666666;">1000</span>, storage_duration))
<span style="color: green; font-weight: bold;">print</span> (<span style="color: #ba2121;">'Storage capacity relative to produced energy </span><span style="color: #bb6688; font-weight: bold;">%.2f%%</span><span style="color: #ba2121;">'</span>
<span style="color: #666666;">%</span> (storage_ratio <span style="color: #666666;">*</span> <span style="color: #666666;">100</span>,))
<span style="color: green; font-weight: bold;">print</span> (<span style="color: #ba2121;">'Share of energy cycled through storage </span><span style="color: #bb6688; font-weight: bold;">%.2f%%</span><span style="color: #ba2121;">'</span>
<span style="color: #666666;">%</span> (stored_energy_ratio <span style="color: #666666;">*</span> <span style="color: #666666;">100</span>))
</pre></div>
<p></p><br />Producing the following output:<div><br /></div><div><div><span style="font-family: courier;">VRES generation potential for Spain: 38.29% of load (13.28% Solar / 25.01% Wind)</span></div><div><span style="font-family: courier;">Energy output 90.318 TWh</span></div><div><span style="font-family: courier;">Storage dimensions: 3.306 TWh @ 17.575 GW (188 h duration)</span></div><div><span style="font-family: courier;">Storage capacity relative to produced energy 3.66%</span></div><div><span style="font-family: courier;">Share of energy cycled through storage 18.75%</span></div></div>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-63223921738161634212022-11-20T14:33:00.014+01:002023-01-25T00:09:33.895+01:00Speculations on Innovation in the Renewable Energy Sector<p><a href="https://en.wikipedia.org/wiki/Experience_curve_effects" target="_blank"></a></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_7wPEHkQ6PprYTi2gSsHHkx9-_ezo8HGV0Tl6KxzhbvNKnq5BB7qYz95tlpAaDJ1VA49zFkbtSv9OuuTD1-NbbNHWoHChNk0vjNG_yAlhiGxfDAlY5BqTrl1s2woxTVAmGWdc4X7weNJ874iOD1q8G7emy0s4xzhKGs7QhB_RpRKYE1Svtp5ztLda-A/s2873/fistful_of_dollars.jpg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="2873" data-original-width="2203" height="453" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_7wPEHkQ6PprYTi2gSsHHkx9-_ezo8HGV0Tl6KxzhbvNKnq5BB7qYz95tlpAaDJ1VA49zFkbtSv9OuuTD1-NbbNHWoHChNk0vjNG_yAlhiGxfDAlY5BqTrl1s2woxTVAmGWdc4X7weNJ874iOD1q8G7emy0s4xzhKGs7QhB_RpRKYE1Svtp5ztLda-A/w347-h453/fistful_of_dollars.jpg" width="347" /></a></div><p style="text-align: left;"><a href="https://en.wikipedia.org/wiki/Experience_curve_effects" target="_blank">Wright's Law</a>, a lesser known cousin of Moore's Law, states, that in most industries, unit-costs decrease by X percent for any doubling of the production volume. In addition, a growing amount of money and attention focused on a particular product tends to increase the opportunity for internal innovation to be successful, specially where there are high degrees of freedom and substitutability without consumers noticing a difference.</p><p style="text-align: left;">Internal innovation tends to be focused on making existing products better or cheaper while external innovation focuses on what customers might actually need or want. Internal innovation is often incremental, low friction and often invisible, while external innovation is often disruptive and requires changes in customer behavior.</p><p style="text-align: left;">While the cost of solar, wind or battery technologies has dropped significantly over the last decades, the current research pipeline shows plenty of potential for significant efficiency improvements.</p><div><div><p></p><p></p><p>If I were a cleantech VC, these are some of the things I would not want to bet against happening over the next few years/decades:</p><p></p><ul><li>Battery chemistry and process innovations might reduce the energy storage cost by 1-2 orders of magnitude, making batteries cost competitive with any current electricity generation technology. Since battery innovation is driven by the need for "light & fast" batteries from transportation use-cases, the outcome might not be quite optimised for stationary applications which can accept "heavy & slow" for the lowest possible price.</li><li>New materials and process innovation for solar cells might increase performance by 50-100% and/or reduce cost by 1-2 orders of magnitude, possibly creating a situation where solar energy becomes too cheap to meter during certain parts of the day.</li><li>Vertical axis wind turbines might become as mature as the current horizontal axis ones. Due to the low center of gravity, they could be particularly useful for floating offshore applications, or with the different visual profile, might also reduce the "Don Quixote effect" of aggressive opposition against onshore wind turbines in some parts of the world.</li><li>As always, commercially viable nuclear fusion is said to be just 15 years away - maybe this time it will be true...</li></ul><p></p><p>(This doesn't mean I have any idea which of the projects in the R&D pipeline are going to pan out, just that there is a good chance that some of them will).</p></div></div>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-84304646430258066192022-11-14T18:13:00.005+01:002023-06-26T11:34:48.832+02:00The case for batch-mode electricity<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGjXY2kmNxz9UtIh6K4iNA5EtHjjkvQ4M_sLZtmsaoUyARqLMjHc7De3uIt_pnoeJFv5_Jfs3PnWbusO37xFq05xYU3kV5q0ynnnm5yjPWP__u0pOBUnZRvV8ggTbiQnh2M72exZDUU1gybelsHu0MpKFw_vKLHUocLxdF-fd95l1FpSi3p50nci4qow/s4032/PXL_20220924_183532077_2.jpg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="3024" data-original-width="4032" height="337" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGjXY2kmNxz9UtIh6K4iNA5EtHjjkvQ4M_sLZtmsaoUyARqLMjHc7De3uIt_pnoeJFv5_Jfs3PnWbusO37xFq05xYU3kV5q0ynnnm5yjPWP__u0pOBUnZRvV8ggTbiQnh2M72exZDUU1gybelsHu0MpKFw_vKLHUocLxdF-fd95l1FpSi3p50nci4qow/w450-h337/PXL_20220924_183532077_2.jpg" width="450" /></a></div>Historically the service provided by electrical utilities has been that of 24/7 instant power, available at the flip of a switch. The keys to providing such a service are power sources which can produce electricity on demand at any time as well as a power grid with plenty of spare capacity for extreme circumstances. In the retail price of electricity, the cost of generation is often a small part (around 5-10 cents) and dwarfed by the cost of the delivery network as well as the cost of over-provisioning in order to ensure a high level of service availability.<p></p><div class="Ar Au Ao" id=":25x"><div aria-controls=":28n" aria-label="Message Body" aria-multiline="true" aria-owns=":28n" class="Am Al editable LW-avf tS-tW tS-tY" g_editable="true" hidefocus="true" id=":25t" itacorner="6,7:1,1,0,0" role="textbox" spellcheck="true" style="direction: ltr; min-height: 376px;" tabindex="1"><div><br /></div><div>The decarbonization of the energy sector is putting pressure on this model from two sides. On one hand, we are replacing fossil fuel power plants which can produce electricity on more or less on demand with wind and solar plants which produce electricity intermittently based on the current availability of wind or sunlight. Wind and solar generators can be combined with sufficient storage into a hybrid power plant which can largely emulate the desired production profile of any traditional power plant - preserving the traditional model of power utilities, at the cost of a lot of additional storage. <a href="https://www.cell.com/joule/fulltext/S2542-4351(19)30300-9" target="_blank">This paper</a> for example shows that if storage costs per unit of energy (kWh) were to fall by 1-2 orders of magnitude, the current model of power grid operations could be maintained at the same levelized scheduled cost as today, but with 100% renewable sources.</div><div><br /></div><div>However, this might not be necessary. With the electrification of transport and heating/cooling, the majority of electricity demand will no longer be instantaneous, real-time and inflexible, but rather asynchronous and somewhat flexible in time.</div><div><br /></div><div>Charging batteries or producing heat requires large amounts of energy to be delivered reliably - however not necessarily right now, but sometime over the next minutes, hours or days. We could rethink and re-design the services provided by the power grid to be more suitable for these more flexible consumers, without any noticeable degradation in the quality of the service seen by the end users. <a href="https://smarten.eu/wp-content/uploads/2022/09/SmartEN-DSF-benefits-2030-Report_DIGITAL.pdf" target="_blank">This study</a> estimates that by 2030, the European power grid could save 60Gw in generation capacity if demand flexibility can be used efficiently. This savings potential corresponds about to the average power consumption of <a href="https://app.electricitymaps.com/zone/FR?aggregated=true" target="_blank">France</a> or <a href="https://app.electricitymaps.com/zone/DE?aggregated=true" target="_blank">Germany</a> today.</div><div><br /></div><div>Many power utilities have some form of time-of-day or even real-time market base variable tariffs to incentivise users to shift their time-flexible usage away from demand peaks. However a simple open-loop control might not be sufficient to unlock the full potential which demand flexibility could have to optimise matching supply and demand in the power grid (<a href="https://web.eecs.umich.edu/~hiskens/publications/05643088.pdf" target="_blank">paper</a>). </div><div><br /></div><div>The answer might instead be two fundamentally different tiers of electricity service - one interactive, real-time and more expensive, where power is always available instantly at the flip of a switch. The other one asynchronous and cheaper, where energy delivery needs to be ordered in advance - to be delivered with some temporal flexibility over minutes, hours or days at the discussion of the network operator, as long as the specifications of the order is respected (e.g. total energy need, time interval, supported range of power levels or interval durations etc.). This kind of forward contracts is similar to how utilities and very large consumers are already reserving their projected electricity needs, anywhere from years in advance to day-ahead.</div><div><br /></div><div>Such a system would require a new control and signalling protocol architecture to be implemented in all the flexible consumers from electric car chargers, building heating and cooling systems, water heaters, potentially all the way to big household appliances. Reaching widespread agreement on such new technical standards is generally a very time consuming and highly political affair.</div><div><br /></div><div>As a communication and distributed systems engineer, my heart obviously goes out to a software based "smart grid" approach which uses sophisticated control mechanisms to optimise the usage of scarce resources. But at this point, I would also not want to bet against innovations in battery chemistry and manufacturing processes to reach the point where cheap storage will allow power grids to continue to operate as they have for decades and making all this fancy new control obsolete again.</div></div></div>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-50401882233340618692022-02-03T18:58:00.008+01:002022-11-06T23:08:44.926+01:00The History of Dehorsification<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi3qNsFLTxgmdIEgM7FymXyq3PuNjWfUvHVMwKHYtC4tSB95OVxRTMBqF3U5I3NwkkPgBTrxa5lPMRhi1pIFCMx6bjyu9opB5qhbLtbp1l4-pEYK8_OedS1QYiGuc_nanWy9pGL-o1MqOdx3tviQIEEGnU88eYCcNSY9jO3aYqdvE_0o8S_n9yEPJ_9eA=s2537" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="2537" data-original-width="2000" height="363" src="https://blogger.googleusercontent.com/img/a/AVvXsEi3qNsFLTxgmdIEgM7FymXyq3PuNjWfUvHVMwKHYtC4tSB95OVxRTMBqF3U5I3NwkkPgBTrxa5lPMRhi1pIFCMx6bjyu9opB5qhbLtbp1l4-pEYK8_OedS1QYiGuc_nanWy9pGL-o1MqOdx3tviQIEEGnU88eYCcNSY9jO3aYqdvE_0o8S_n9yEPJ_9eA=w286-h363" width="286" /></a></div>De-carbonization will require in particular a radical transformation of the energy and transportation sectors over the next few decades. Many people find this daunting or fear that it might be not even be possible in such a time-frame.<p></p><div>Looking at the <a href="https://thetyee.ca/News/2013/03/06/Horse-Dung-Big-Shift/">history of de-horsification,</a> a similarly radical transformation of the transportation sector in the first half of the 20th century shows that such things are indeed possible, but can be tricky, disruptive and take a long time.</div><div><br /></div><div>At the beginning of the 20th century, the horse was undoubtedly the workhorse of urban transportation for both goods and people, while the new horseless carriages were expensive and unreliable playthings for crazy, rich people. Less than 50 years later the roles are reversed and most horses today likely travel more miles standing in a trailer pulled by a SUV than on their own feet. In 1900, horses were very common - nearly as common per capita as cars are today. New York City counted over 200'000 working horses in 1900, today there are just a few dozen - including those of the NYPD mounted police.</div><div><br /></div><div>The reasons why millions of horses were replaced by millions of cars and trucks over the course of a few decades is somewhat disputed - likely that cars somehow presented an improvement overall.</div><div><br /></div><div>Initially, cars were much more expensive than their horse powered equivalents, but according to <a href="https://lancasteronline.com/news/car-or-buggy-which-is-cheaper-to-drive/article_5586626b-d27a-537d-a690-6bfff5d93c21.html">these claims</a>, even today they are slightly more expensive.</div><div><br /></div><div><a href="https://www.blog.greenprojectmanagement.org/index.php/2019/05/13/pollution-why-we-replaced-horses-with-automobiles/">Some argue</a> that economic arguments were not the primary motivation, but the growing piles of bio-waste threatening to overwhelm the rapidly growing urban areas. <a href="https://nautil.us/did-cars-save-our-cities-from-horses-1261/">Others argue</a> that the environmental crisis theory is likely a bunch of horse-sh*t...</div><div><br /></div><div>Undoubtedly the transformation required and/or triggered massive changes in infrastructure as well as to the fabric of society. Thousands of horse stables disappeared from city streets and or were replaced by parking garages and gas stations. Entire professions like wainwrights, teamsters, blacksmiths or horse breeders became irrelevant over the course of a few decades. A rather decentralized industry of horse breeding, carriage making and growing horse feed was replaced by a highly concentrated automotive and fossil fuel industry. Fortunes were made and lost as part of sweeping socio-economic changes, causing plenty of excitement and anguish and spanning a period of great turmoil with two world wars and a major economic crisis.</div><div><br /></div><div>While this particular history shows that such radical transformations of entire sectors are possible and have happened in the recent past, it does not provide any guidance on whether they can be accelerated or whether the associated disruptions to peoples lives can be easily avoided.</div><div><br /></div><div></div>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-79710976098812387722022-01-20T15:50:00.011+01:002022-01-22T17:50:58.621+01:00Lessons from the Pandemic: Tragedy of the Commons vs. Social Contracts<p></p><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhNb161dXYB63i9PJhjvlDU20ra_DXT6sHh0gQAEoqjPJ4Gezway5W-Ig5IIKK6wxMqMj193r5mnWXf8KbZ3c_RVK89ltbNgZ0LlJ3adFkgBqJ-yQVMt9GiygAIUNZo_x_WPPE9Lonyr7UrbMussjkDCOuYBV4WRkClCZWKtt2X0Q0jAXw_pHgHZxDgtQ=s751" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="751" data-original-width="637" height="388" src="https://blogger.googleusercontent.com/img/a/AVvXsEhNb161dXYB63i9PJhjvlDU20ra_DXT6sHh0gQAEoqjPJ4Gezway5W-Ig5IIKK6wxMqMj193r5mnWXf8KbZ3c_RVK89ltbNgZ0LlJ3adFkgBqJ-yQVMt9GiygAIUNZo_x_WPPE9Lonyr7UrbMussjkDCOuYBV4WRkClCZWKtt2X0Q0jAXw_pHgHZxDgtQ=w329-h388" width="329" /></a></div>The recent pandemic has provided a lot of insights into how societies and governments around the world respond to the kind of collective crisis, where the potential harm faced by each member of the community does not primarily depend on their direct actions, but also on the actions of anybody around them.<p></p><p>We can hope to draw some lessons from these insights for the the climate crisis, which presents similar challenges but over a much longer time and at global scale.</p><p>On the bright side, lessons from the pandemic have shown that eventually most governments and communities have eventually responded quite decisively and to a level that might not have been expected. </p><p>One specific episode that happened in Switzerland around June 6th 2020, I found particularly insightful: the question of wearing masks in public transport.</p><p>In the days leading up to June 6th, an estimated less than 5% of passengers in public were wearing masks. Despite the government having strongly recommended it and repeatedly appealed to the individual responsibility ("Eigenverantwortung - the favorite word of the Swiss government during the whole pandemic). Polls had shown that around three quarters of the population would be in favour of a mandate to wear masks in public transport. Yet almost nobody was wearing one themselves...</p><p>On June 1st the Swiss federal executive <a href="https://www.admin.ch/gov/en/start/documentation/media-releases.msg-id-79711.html">issued a statement</a> that wearing masks in public transport shall be mandatory for anybody over the age of 12 starting on Monday June 6th. And suddenly on the morning of June 6th, a vast majority of public transport users (estimated over 95%) started wearing masks and have done so ever since, despite no particular active enforcement of this new rule.</p><p>According to expert consensus at the time, the main effect of wearing a simple surgical mask is to hold back droplets in order not to infect others and less so to protect the wearer from being infected. I.e. we are best protected if everybody else around us is wearing a mask.</p><p>However wearing a mask comes at some personal cost and inconvenience. Masks cost money - about 10c per piece, they are uncomfortable to wear, fog up glasses and maybe most significantly for western societies, wearing a mask in public carries a social cost of awkwardness and violating cultural norms.</p><p>Such a conflict between personal cost and collective benefit - or personal benefit at the expense of the community is illustrated by the popular metaphor of the<a href="https://en.wikipedia.org/wiki/Tragedy_of_the_commons"> tragedy of the commons</a>.</p><p>This episode is a stark reminder of how ineffective even the most passionate appeals to reason and responsibility are, if self interests conflicts even the tiniest bit (10 cents per mask!) with the common good. Even if the majority of the population believes it would be right thing to do and something should be done.</p><p>Yet the simple act of the government changing a strong recommendation to a rule has shifted behaviour massively over night - from less than 5% to more than 95% compliance. All this without without much material difference and enforcement beyond passive-aggressive peer pressure for which the Swiss are so famous for.</p><p>I think this is a hopeful example how a collective agreement to cooperate and be reasonable is essential to overcoming to overcoming the tragedy of the commons. As the example shows, a strong symbolic signal of agreement can massively shift behaviour literally overnight.</p><p>Some kind of social contract, formally or informally, can help to overcome selfishness in favour of the common common good if we somehow convince ourselves that we are all going to do it and I won't be the only idiot in the end holding the short end of the straw. Such agreement can take many forms - from social norms, government laws, religious doctrine or moral codes - whatever works best for a given community.</p><p>However it does not seem to be enough that most of us think it's a good idea and we should all do it, there has to be a moment when a social contract is forged and the "volonté de tous" becomes the "volonté générale"</p><p>The lesson from this episode might be that our best chance to solve the climate crisis is that unless self interest becomes become fully aligned with the common good (i.e the "<a href="https://www.gatesnotes.com/Energy/Introducing-the-Green-Premiums">green premium</a>" has to become null or negative) , we need to move from appeals for individual voluntary actions to collective agreements - e.g. binding rules and regulations on a global scale. And that might just be the hardest part of it.</p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-62279469420312521842022-01-03T22:20:00.016+01:002022-01-20T19:23:07.648+01:00Lessons from the Pandemic: The Bias towards Inaction<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgonwdc1X9uH2_wV2eQkayFGzdJCE4tjiTAnfIYv5M3QpMM1gZwWI7Pt6lK2iXMbuhVBcIM34bQWynv1g3LLPQeIeMwUSMYE834VirGx-XIpc8z6vKiD6-5V1FecNw5pUfFOi1AXd1vccSDMQkUPO_Ah3vI4hDA87Is8cmvzwRMuAgCveYEleK0Li8DIA=s1365" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1365" data-original-width="1024" height="430" src="https://blogger.googleusercontent.com/img/a/AVvXsEgonwdc1X9uH2_wV2eQkayFGzdJCE4tjiTAnfIYv5M3QpMM1gZwWI7Pt6lK2iXMbuhVBcIM34bQWynv1g3LLPQeIeMwUSMYE834VirGx-XIpc8z6vKiD6-5V1FecNw5pUfFOi1AXd1vccSDMQkUPO_Ah3vI4hDA87Is8cmvzwRMuAgCveYEleK0Li8DIA=w322-h430" width="322" /></a></div><div>In the context of the climate crisis, I have been wondering why we have so far not been acting more decisively to prevent what could credibly turn into the largest civilisatory catastrophe in recent human history (at least in absolute terms). </div><div><br /></div><div>The pandemic has provided an opportunity to observe the rapid unfolding of a wide range of collective and government crisis responses, that could help to identify some pattern.</div><div><br /></div>The <a href="https://en.wikipedia.org/wiki/Trolley_problem" style="color: #1155cc;">trolley problem</a> is a famous thought experiment to illustrate an ethical dilemma where the needs of the many might outweigh the needs of the few : a trolley is assumed to be barreling down the track towards a group of people that it would most likely hit while a bystander has the opportunity to move a switch, which will redirect the trolley on another track, where it will likely hit a single other person.<p></p><div>While moral philosophers have a <a href="https://plato.stanford.edu/entries/doing-allowing/">more nuanced perspective on the moral equivalence of action vs. inaction, of doing vs. allowing</a>, the laws and norms of most communities, typically give a very simple recommendation: just don't!</div><div><br /></div><div>If the bystander were to act, they would likely be charged with unauthorised tampering with railroad equipment and involuntary manslaughter at the least. If they don't act, they would likely not be blamed for anything and receive many thoughts and prayers for having to witness such an unfortunate tragedy.</div><div><br /></div><div>In response to the current pandemic or the climate crisis, we can see a similar hesitation to act from politicians or even public opinion - even if the scientific consensus is solid. "Pay or suffer some more now to avoid paying or suffering even more in the future" does not make a solid plank for a re-election platform.</div><div><br /></div><div>Most societies and liberal democracies in particular tend to weight negative rights and freedoms higher than positive ones. I.e. the right to be free from interference and infringement vs. the right to receive support. Which means that governments are typically much more hesitant to infringe on one persons freedom to satisfy another's right to something - e.g. restricting one person's right to party in order to protect another persons right to life and health.</div><div><br /></div><div>In addition, there might be doubt about the certainty and gravity of the outcomes. Sure, the scientific consensus indicates that the people on the trolley track will most certainly (99.7%?) be hurt seriously. But in times of crisis we often find strengths in <a href="https://en.wikipedia.org/wiki/Optimism_bias" style="color: #1155cc;">unjustified optimism</a> or even <a href="https://en.wikipedia.org/wiki/Magical_thinking" style="color: #1155cc;">magical thinking</a>. Many good stories are told of victory snatched from the jaws of defeat at the last instant...</div><div><br /></div><div>Those who act, can and will be blamed for the consequences of their actions, especially when things turn out badly. And they will possibly get no credit for the harm they might have just prevented - after all <a href="https://en.wikipedia.org/wiki/Self-defeating_prophecy" style="color: #1155cc;">it did not happen in the end</a>! And except for some <a href="https://en.wikipedia.org/wiki/Duty_to_rescue" style="color: #1155cc;">well defined situations</a>, inaction rarely leads to serious blame.</div><div><br /></div><div>Hence inaction is a rational default behavior, in particular for those who care about public opinion, like for example elected officials. Which means that when faced with an unprecedented crisis which requires unpopular or painful measures, governments have a natural tendency to do too little too late - at least until they believe strongly that these measures will be expected, supported and even demanded by a majority of the electorate.</div><div><br style="background-color: white; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: small;" /></div><div>Creating a mandate for action would require to establish a consensus on tight causality between actions and inactions respectively to the corresponding outcomes in order to attribute credit or blame. Strong mandates to act in order to prevent harm are rare and exist for example in maritime law with the obligation to assist those in distress at sea. Shipwrecks are a pretty well known and routine type of disaster and the current legal framework had as catalyst a particularly famous one: the sinking of the Titanic.</div><div><br /></div><div>To overcome our natural aversion to change and <a href="https://en.wikipedia.org/wiki/Status_quo_bias">preference for the status quo</a>, a rule of thumb in the startup community is that any new alternative needs to be 10x more compelling somehow. Which means that in order for communities and political leaders to choose action over inaction, the outcomes need not only be obvious and threatening, but overwhelmingly so.</div><div><br /></div><div>In the meantime, we will continue to "<a href="https://www.youtube.com/watch?v=Op_v2PHDn-0" target="_blank">sit tight and assess</a>"...</div>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-91497323502147603622021-04-16T14:13:00.002+02:002021-04-16T17:20:38.462+02:00Entgendern nach Phettberg<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM_ldepUQeZclxC_U-VCfOP2GRlf-LdrgzWwCvg3EyzXG2DH3e3zUb-2FXbdm-bSlVQsbv1BMKTQj_k308JiMtJRf2Cjz6kY9EuTm-4WTUzhZUGjR4tsG6GEsS3Q9HxpDNYwnAyIp_Yx_k/s1024/PXL_20210416_101433871.jpg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="655" data-original-width="1024" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM_ldepUQeZclxC_U-VCfOP2GRlf-LdrgzWwCvg3EyzXG2DH3e3zUb-2FXbdm-bSlVQsbv1BMKTQj_k308JiMtJRf2Cjz6kY9EuTm-4WTUzhZUGjR4tsG6GEsS3Q9HxpDNYwnAyIp_Yx_k/s320/PXL_20210416_101433871.jpg" width="320" /></a></div>Liebe Lesys,<p></p><p>Ich möchte auch auf Deutsch gerne inklusiv schreiben, ohne dass jeder Satz nach identitätspolitischem Kulturkampf schreit. Mein Vorzug wäre ein grammatisches Neutrum für Personenbezeichnungen, das nicht zu bürokratisch tönt ("Zielpersonen des staatlichen Bildungsauftrages") oder semantisch ungenau ist, wie <a href="https://de.wikipedia.org/wiki/Geschlechtergerechte_Sprache#Substantivierte_Partizipien_oder_Adjektive">substantivierte Partizipien</a> ("Lernende" oder "zur Schule Gehende").</p><p>Dieser Vorschlag (<a href="https://www.researchgate.net/publication/343974830_Entgendern_nach_Phettberg_im_Uberblick">link</a>) des Germanistys Thomas Kronschläger für eine neue Klasse von genderneutralen Personenbezeichnungen ist mir bisher am sympathischsten:</p><p>Aus dem Stamm eines bestehenden Nomen und einer neuen Endung auf -y für Singular und -ys für Plural entsteht ein neues generisches Neutrum (z.B "das Schüly", "die Schülys").</p><p>Diese neue Y-Form erinnert auch irgendwie an den Diminutiv im Schweizerdeutschen, den viele Deutsche scheinbar niedlich finden. Und dieser Jöh-Faktor könnte vielleicht auch der neutralen Y-Form im deutschsprachigen Raum zum Durchbruch verhelfen.</p>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-52385871141293431222020-02-23T01:03:00.006+01:002022-01-30T22:31:06.753+01:00Career Development for Senior Engineers on the technical ladder<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF9RCnSDd6fdVeknz_TwQLAEzTmGPszsGzsR0SXxVcv4VsnxVJCj5XVEIva8jKz_AOtm2aIYtV03JjKoM5zhzXL_4kAe8frsr5pJDQkb8Gx6LaoDAet81TJxvzxZaaaPlpAXDUSy_kK3He/s1600/13+-+2.jpg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1414" data-original-width="1098" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF9RCnSDd6fdVeknz_TwQLAEzTmGPszsGzsR0SXxVcv4VsnxVJCj5XVEIva8jKz_AOtm2aIYtV03JjKoM5zhzXL_4kAe8frsr5pJDQkb8Gx6LaoDAet81TJxvzxZaaaPlpAXDUSy_kK3He/s320/13+-+2.jpg" width="248" /></a></div>
As part of the yearly performance review process, we are supposed to describe what we do and how we add value to the organisation.<br />
<br />
I am a software engineer by training and once prided myself on being a pretty decent programmer. At this point in my career, I am in a fairly senior position on the technical ladder at a large innovation driven tech company. This means that I now write a lot less code than most people in my team. The dual-ladder career system allows for formal career advancement of employees in technical roles without having to necessarily change into management roles - but still expects similar levels of strategic impact.<br />
<br />
So what do I actually do?<br />
<br />
When moving through levels of seniority, my focus has shifted from writing code to reviewing code, then from writing design documents to commenting on other people's designs. I still remember succinctly the advice of a more senior colleague, that the key to getting promoted to the next level would be by becoming comfortable with taking credit for other people's work. Even as an "individual contributor". While sounding snarky, this is very sound advice, specially as this does not come naturally to many engineers who idealize self-reliance and who think they could do everything by themselves if they just had enough time.<br />
<br />
While critiquing, guiding and mentoring others - and ultimately taking some credit for their accomplishments - helped me to get to the current level, it won't be enough for the next level.<br />
<br />
So what do I actually do?<br />
<br />
Maybe my most impactful contribution these days is actually in storytelling. Developing narratives, analogies, metaphors and mental models to describe things that are abstract, unobvious and worst of all - don't yet exist. Creating the vocabulary to help an organisation to talk about what it is that we should be doing and how we could get there.<br />
<br />
After years of honing the skills and craft of software and systems development, it comes down to the oldest cultural competence - the one that makes us the most distinctively human: the ability to imagine and talk about things that don't exist. And through that, maybe help them to become real...<br />
<br />Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-27649003688124636832019-04-20T23:14:00.000+02:002019-04-22T21:26:00.547+02:00Email to Disaspora* posting Bot<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPAEsiK38pdder4ytRxPfA6SPQBR2JZQ_6Pmp7_A3HuDGwSnCLoCIm-7x0xtNylFC_CerUihp_96CfxewXxcr6iG5VktoQ-lL1l6pfSzyt1DSzbdJh9VwIHsmjJKepow36GroQsoj4_se2/s1600/IMG_4027.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1321" data-original-width="1600" height="264" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPAEsiK38pdder4ytRxPfA6SPQBR2JZQ_6Pmp7_A3HuDGwSnCLoCIm-7x0xtNylFC_CerUihp_96CfxewXxcr6iG5VktoQ-lL1l6pfSzyt1DSzbdJh9VwIHsmjJKepow36GroQsoj4_se2/s320/IMG_4027.JPG" width="320" /></a></div>
What I still miss the most after moving from G+ to Diaspora* for a my casual public social network posting is a well integrated mobile app for posting on the go.<br />
<br />
The main use-case for me is posting photos on the go, which I now mostly take on my cellphone and minimally process with Google Photos.<br />
<br />
One of the problems with the mobile app for Diaspora* (Dandelion in the case of Android) is that the size limit for photo uploads is quite small compared to the resolution of todays cellphone cameras. There is also not much point of uploading high-resolution images for purely on-screen consumption to an infrastructure managed by volunteers on a shoestring budget. I also liked the ability to geo-tag the mobile posts by explicitly selecting a nearby landmark to obfuscate a bit the current location.<br />
<br />
For a few weeks now, I have been sharing my account with a <a href="https://blog.kugelfish.com/2018/12/google-migration-part-viii-export-to.html">G+ archive bot</a> that is uploading recycled posts from the takeout archive (see <a href="http://blog.kugelfish.com/2018/10/google-migration-part-i-takeout.html">here for the first part of the series</a> describing the process). I like the structured formatting and meta-data tags that come from automated processing and since my bot seems to be getting more likes that I do, I am thinking why not keep it around?<br />
<br />
I am a heavy email user and email clients are well integrated into the sharing functions of both Android and IOS mobile platforms. Since the posting bot is already using a free web-mail account for error reporting it would be easy to use the same account for sending emails to the bot for post-processing and posting. Only emails originating from my own address(es) should be converted into a post. Thanks to <a href="https://de.wikipedia.org/wiki/DomainKeys">DKIM</a> domain authentication used by most major email providers today, we can somewhat trust the authenticity of the sender information in the header.<br />
<br />
This new bot is using the <a href="https://de.wikipedia.org/wiki/Post_Office_Protocol">POP3 protocol</a> to access the inbox of the online hosted email account, download the emails, check the senders and extract the plain text and image attachment parts in particular. If available, Exif GPS data is extracted from the images and reverse-geocoded using OpenStreetMap to the rough neighborhood of where the image was taken (<a href="https://blog.kugelfish.com/2019/04/extracting-location-information-from.html">see previous post</a>). The images are rotated and scaled to a maximum size for upload. Some simple, hard-coded "business rules" are used to generated additional hashtags for some of common use-cases - primarily photo sharing or link sharing.<br />
<br />
The post is then staged the same format and directory structure as for the <a href="https://blog.kugelfish.com/2018/12/google-migration-part-vii-conversion.html">takeout archive processor</a> so that the <a href="https://blog.kugelfish.com/2018/12/google-migration-part-viii-export-to.html">same posting bot</a> can be re-used.<br />
<br />
Similarly, we can run the new the combination of email processor and diaspora exporter from the crontab on a Raspberry Pi or some other linux based always-on server platform:<br />
<pre style="background-color: #eeeeee; border: 1px dashed #999999; color: black; font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><code>PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
19 * * * * /home/pi/mail_bot/mail_bot.sh</code></pre>
Where the <span style="font-family: "courier new" , "courier" , monospace;">mail_bot.sh</span> script is as follows:<br />
<pre style="background-color: #eeeeee; border: 1px dashed #999999; color: black; font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><code>#!/bin/sh
cd /home/pi/mail_bot
./mail_bot.py --login-info=./logins.json --staging-dir=./staging --mail-errors
/home/pi/post_bot/post_bot.py --staging-dir=./staging --login-info=./logins.json --mail-errors</code></pre>
The email processing component is in <span style="font-family: "courier new" , "courier" , monospace;">mail_bot.py</span> below. It depends on the module exif2hashtag.py from the previous post as well as on the additional packages dateutil, dkimpy and PIL/Pillow, which can again be installed as <span style="font-family: "courier new" , "courier" , monospace;">pip3 python-dateutil dkimpy Pillow</span>.<br />
<br />
The mail section in the <span style="font-family: "courier new" , "courier" , monospace;">logins.json</span> file requires two additional <span style="font-family: "courier new" , "courier" , monospace;">'pop-server'</span> with the name or address of the email accounts pop3 service and <span style="font-family: "courier new" , "courier" , monospace;">'authorized-senders' </span>with a list of email addresses wholes messages will be transformed into Diaspora* posts.<br />
<pre style="background-color: #eeeeee; border: 1px dashed #999999; color: black; font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><code>#!/usr/bin/env python3
import argparse
import datetime
import email
from email.mime.text import MIMEText
import io
from io import StringIO
from io import BytesIO
import json
import logging
import logging.handlers
import os
import poplib
import shutil
import smtplib
import sys
import dateutil.parser
import dkim
import html2text
import PIL.Image
import exif2hashtag
ISO_DATE = '%Y%m%d'
ISO_DATETIME = ISO_DATE + '_%H%M%S'
# Extra hashtags for the sites I might be posting links from a mobile reader.
SITES = {
'www.republik.ch' : ['Republik', 'News', 'media', 'lang_de', 'CH', 'Switzerland'],
'www.tagesanzeiger.ch' : ['Tagesanzeiger', 'news', 'media', 'lang_de', 'CH', 'Switzerland'],
'www.youtube.com' : ['YouTube'],
'wikipedia.org' : ['Wikipedia'],
'blog.kugelfish.com' : ['Blog', 'mywork', 'CC-BY', 'technology', 'programming'],
}
def send_error_message(txt, email_info):
"""Send a crash/error message to a configured email address."""
server = smtplib.SMTP(email_info['smtp-server'])
server.starttls()
server.login(email_info['username'], email_info['password'])
msg = MIMEText(txt)
msg['From'] = email_info['username']
msg['To'] = email_info['recipient']
msg['Subject'] = 'error message from %s on %s' % ('mail-bot', os.uname()[1])
server.sendmail(email_info['username'], email_info['recipient'], msg.as_string())
server.quit()
def validate(authorized_senders, sender, msg):
"""Check DKIM message signature and whether message is from an approved sender."""
if not dkim.verify(msg):
return False
for s in authorized_senders:
if s in sender:
return True
return False
def header_decode(hdr):
"""Decode RFC2047 headers into unicode strings."""
str, enc = email.header.decode_header(hdr)[0]
if enc:
return str.decode(enc)
else:
return str
def export_image(img, outdir, num, max_size):
"""Reformat and stage image for posting to diaspora."""
exif_info = exif2hashtag.get_exif_info(img)
gps_info = exif2hashtag.get_gps_info(exif_info)
latlon = exif2hashtag.get_latlon(gps_info)
orientation = exif_info.get('Orientation', None)
if orientation:
if orientation == 3:
img=img.rotate(180, expand=True)
elif orientation == 6:
img=img.rotate(270, expand=True)
elif orientation == 8:
img=img.rotate(90, expand=True)
destination = os.path.join(outdir, 'img_%d.jpg' % num)
source_size = max(img.size[0], img.size[1])
if max_size and source_size >= max_size:
scale = float(max_size) / float(source_size)
img = img.resize((int(img.size[0] * scale), int(img.size[1] * scale)), PIL.Image.LANCZOS)
img.save(destination, 'JPEG')
return exif2hashtag.get_location_hashtags(latlon)
def export_message(msg, outdir, image_size):
"""Stage message for posting to diaspora."""
hashtags = ['mailbot']
content = []
title = header_decode(msg.get('Subject'))
if title:
content.append('### ' + title)
content.append('')
img_count = 0
for part in msg.walk():
if part.get_content_type() == 'text/html':
txt = part.get_payload(decode=True).decode("utf-8")
for str, tags in SITES.items():
if str in txt:
hashtags.extend(tags)
converter = html2text.HTML2Text()
converter.ignore_links = True
converter.body_width = 0
content.append(converter.handle(txt))
elif part.get_content_type() == 'text/plain':
elif part.get_content_type() == 'image/jpeg':
img_count += 1
data = BytesIO()
data.write(part.get_payload(decode=True))
data.seek(0)
img = PIL.Image.open(data)
for tag in export_image(img, outdir, img_count, image_size):
if not tag in hashtags:
hashtags.append(tag)
if img_count > 0:
hashtags = ['photo', 'photography', 'foto', 'myphoto', 'CC-BY'] + hashtags
if hashtags:
content.append(' '.join(('#' + tag for tag in hashtags)))
content_file = io.open(os.path.join(outdir, 'content.md'), 'w', encoding='utf-8')
content_file.write('\n'.join(content))
content_file.close()
#---------------------
parser = argparse.ArgumentParser(description='Coolect post images referenced from a set of posts')
parser.add_argument('--staging-dir', dest='staging_dir', action='store', required=True)
parser.add_argument('--login-info', dest='login_info', action='store', required=True)
parser.add_argument('--image-size', dest='image_size', action='store', type=int, default=1024)
parser.add_argument('--mail-errors', dest='mail', action='store_true')
args = parser.parse_args()
# Set up logging to both syslog and a memory buffer.
log_buffer = StringIO()
logging.basicConfig(stream=log_buffer, level=logging.INFO)
logging.getLogger().setLevel(logging.INFO)
syslog = logging.handlers.SysLogHandler(address='/dev/log')
syslog.setFormatter(logging.Formatter('diaspora-mail-bot: %(levelname)s %(message)s'))
logging.getLogger().addHandler(syslog)
try:
# Load login/authentication data from a separate file.
login_info = json.load(open(args.login_info))
email_info = login_info['mail']
pop3 = poplib.POP3_SSL(email_info['pop-server'])
pop3.user(email_info['username'])
auth = pop3.pass_(email_info['password'])
msg_count = pop3.stat()[0]
logging.info('%d new messages on %s' % (msg_count, email_info['pop-server']))
for msg_num in range(1, msg_count + 1):
msg_txt = b'\n'.join(pop3.retr(msg_num)[1])
msg = email.message_from_bytes(msg_txt)
sender = msg.get('From')
subject = msg.get('Subject')
if not validate(email_info['authorized-senders'], sender, msg_txt):
logging.info('dropping message from unauthorized sender "%s" - subject: "%s"' % (sender, subject))
pop3.dele(msg_num)
continue
timestamp = dateutil.parser.parse(msg.get('Date'))
outdir = os.path.join(args.staging_dir, timestamp.strftime(ISO_DATE), timestamp.strftime(ISO_DATETIME))
if not os.path.exists(outdir):
os.makedirs(outdir)
try:
export_message(msg, outdir, args.image_size)
pop3.dele(msg_num)
except:
logging.info('error exporting msg %d - deleting directory %s' % (msg_num, outdir))
shutil.rmtree(outdir, ignore_errors=True)
raise
pop3.quit()
except (KeyboardInterrupt, SystemExit):
sys.exit(1)
except Exception as e:
logging.exception('error in main loop')
if args.mail and 'mail' in login_info:
send_error_message(log_buffer.getvalue(), login_info['mail'])
sys.exit(1)
</code></pre>
<br />Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-62683796758719073352019-04-18T22:53:00.000+02:002019-04-18T22:53:41.464+02:00Extracting location information from Photos<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRkSbvjwU3KgIWwd_Geiuergy-3bqmk9lGQh57XYQt2Ja65Rsim6Nc_DRWJNPGQ_LgclcXXT9DIN9W6vq0xEGSCRPPP_oJGw7kSMts7onyDq8RS5H6EllbPeGrfj6rNMSXtlUM_FhxrNva/s1600/newhampshire+093.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRkSbvjwU3KgIWwd_Geiuergy-3bqmk9lGQh57XYQt2Ja65Rsim6Nc_DRWJNPGQ_LgclcXXT9DIN9W6vq0xEGSCRPPP_oJGw7kSMts7onyDq8RS5H6EllbPeGrfj6rNMSXtlUM_FhxrNva/s320/newhampshire+093.jpg" width="320" /></a></div>
Photos exported from digital cameras often contain meta-data in Exif format (Exchangeable Image File Format). For images taken with cellphone cameras, this info typically also includes (GPS) location information of where the photo was taken.<br />
<br />
Inspired by<a href="https://blog.kugelfish.com/2018/11/google-migration-part-vi-location.html"> this previous post </a>on the mapping of GPS lat/lon coordinates from Google+ location data to a rough description of the location, we could also use the location encoded in the photo itself.<br />
<br />
We are using again the <a href="https://wiki.openstreetmap.org/wiki/Nominatim#Reverse_Geocoding">reverse geocoding service</a> from OpenStreetMap to find the names of the country and locality in which the GPS coordinates are included in.<br />
<br />
For the purpose of public posting, reducing the accuracy of the GPS location to the granularity of the city town or village provides some increased confidentiality of where the picture was taken compared to the potentially meter/centimeter resolution accuracy of GPS data that generally allows to pinpoint the location down to a building and street address.<br />
<br />
Fractional numbers are represented as ratios of integers in Exif. For example the number 0.5 could be encoded as the tuple (5, 10). The coordinates in the Exif location meta-data are represented in the DMS (Degrees Minutes Seconds) format which needs to to be converted into the <a href="https://en.wikipedia.org/wiki/Decimal_degrees">DD (decimal degree)</a> format used by most GIS systems including OpenStreetMap.<br />
<br />
<br />
<pre style="background-color: #eeeeee; border: 1px dashed #999999; color: black; font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><code>#!/usr/bin/env python
import sys
import geopy
import PIL.Image
import PIL.ExifTags
import pycountry
geocoder = geopy.geocoders.Nominatim(user_agent='gplus_migration', timeout=None)
def get_location_hashtags(latlon):
"""Reverse geo-code lat/lon coordinates ISO-code / country / municipality names."""
hashtags = []
if latlon:
addr = geocoder.reverse((latlon[0], latlon[1])).raw
if 'address' in addr:
addr = addr['address']
cc = addr['country_code'].upper()
hashtags.append(cc)
hashtags.append(pycountry.countries.get(alpha_2=cc).name.replace(' ',''))
for location in ['city', 'town', 'village']:
if location in addr:
hashtags.append(addr[location].replace(' ', ''))
return hashtags
def get_exif_info(img):
"""Decode Exif data in image."""
ret = {}
info = img._getexif()
if not info:
return ret
for tag, value in info.items():
decoded = PIL.ExifTags.TAGS.get(tag, tag)
ret[decoded] = value
return ret
def get_gps_info(info):
"""Decode GPSInfo sub-tags in Exif data."""
ret = {}
if not info or not 'GPSInfo' in info:
return ret
for tag, value in info['GPSInfo'].items():
decoded = PIL.ExifTags.GPSTAGS.get(tag, tag)
ret[decoded] = value
return ret
def degrees_from_ratios(ratios):
"""Convert from Exif d/m/s array of ratios to floating point representation."""
f = [(float(r[0]) / float(r[1])) for r in ratios]
return f[0] + f[1] / 60.0 + f[2] / 3600.0
def get_latlon(gps_info):
"""Extract the GPS coordinates from the GPS Exif data and convert into fractional coordinates."""
lat = gps_info.get('GPSLatitude', None)
lat_hemi = gps_info.get('GPSLatitudeRef', None)
lon = gps_info.get('GPSLongitude', None)
lon_hemi = gps_info.get('GPSLongitudeRef', None)
if lat and lat_hemi and lon and lon_hemi:
return (degrees_from_ratios(lat) * (-1 if lat_hemi == 'S' else 1),
degrees_from_ratios(lon) * (-1 if lon_hemi == 'W' else 1))
else:
return None
def get_camera(info):
"""Get Camera make & model as another example of Exif data."""
if 'Make' in info and 'Model' in info:
return '%s %s' % (info['Make'], info['Model'])
else:
return None
#------------------------------------------------------
for filename in sys.argv[1:]:
image = PIL.Image.open(filename)
exif_info = get_exif_info(image)
gps_info = get_gps_info(exif_info)
latlon = get_latlon(gps_info)
print ('%s : %s %s' % (filename, get_camera(exif_info), get_location_hashtags(latlon)))
</code></pre>
<br />Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-56401636037741397772018-12-28T14:06:00.000+01:002018-12-28T14:06:14.119+01:00The Fallacy of distributed = good<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYCc8-c3avF6EDiaG45TxvtpE-Avi3nyadYYZai2tjfKk9VS1qotxkW_mWxFiV2F3OA0dBHmpfxybxGo3TG-40z4fPUGGDTvthsPrbFK7Va9xPpB2PXuwqsJeW9_rOyvY4LYZVBUyuFCtX/s1600/IMG_2688.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="945" data-original-width="1600" height="236" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYCc8-c3avF6EDiaG45TxvtpE-Avi3nyadYYZai2tjfKk9VS1qotxkW_mWxFiV2F3OA0dBHmpfxybxGo3TG-40z4fPUGGDTvthsPrbFK7Va9xPpB2PXuwqsJeW9_rOyvY4LYZVBUyuFCtX/s400/IMG_2688.JPG" width="400" /></a></div>
I have recently been looking for an alternative social media platform and started using Diaspora* via the <a href="http://diasporing.ch/">diasporing.ch</a> pod. Not unlike the cryptocurrency community, the proponents of the various platforms in the <a href="https://en.wikipedia.org/wiki/Fediverse">Fediverse</a> seem to rather uncritically advocate the distributed nature of these platforms as an inherently positive property in particular when it comes to privacy and data protection.<br />
<br />
I tend to agree with Yuval Harari who argues in <i><a href="https://en.wikipedia.org/wiki/Sapiens:_A_Brief_History_of_Humankind">"Sapiens"</a> </i> that empires or scaled, centralized forms of organization are one of Homo Sapiens' significant cultural accomplishments. A majority of humans through history have lived as part of some sort of empire. Empires can provide prosperity and ensure lasting peace and stability - like the <a href="https://en.wikipedia.org/wiki/Pax_Romana">Pax Romana</a> or in my generation, the Pax Americana. We often have a love/hate relationship with empires - even many protesters who are busy burning American flags during the day, secretly hope that their children some day will get into Harvard and have a better life. Libertarians seem to think of a land without much central governance as a place where strong individuals can realize their dreams of freedom and prosperity - like the romanticized frontier world of daytime TV westerns. My cynical self imagines rather a kind of post-apocalyptic Mad Max world, where our school bullies become sadistic local warlords. In European history, the thousand years after the fall of the Roman Empire, which featured highly distributed power structures are called the <i>dark</i> middle ages for good reasons.<br />
<br />
In the online world, many of us love to hate the big social media platforms and yet <a href="https://www.statista.com/statistics/272014/global-social-networks-ranked-by-number-of-users/">billions of us return there every month</a>. Maybe because this is where our friends, family and everybody else are as well or because they generally offer a smooth and polished service or are good a giving us what we want? When it comes to security, the largest platforms can afford to invest more in security and have impressively competent security and operations teams to protect our data from being compromised. As the Roman legions, they do not always succeed and their failures are highly publicized. But more often than not, they succeed and looking at it dispassionately, our data is probably nowhere as save as with one of the large providers of online or cloud services. Yes, the large platforms are driven by commercial interests, which also makes them predictable as they have a lot to loose and tend to follow laws with pedantic sophistication.<br />
<br />
I fail to see how a distributed architecture alone should inherently improve privacy or data protection. For most of us in the Fediverse, our pod-admins rather than us are de facto in possession and control of our data. De jure, they don't have to worry much about data protection and privacy laws because they are too small to be on the radar of any regulatory agency. Pods can disappear from the network at any time without warning and account migration between pods is generally not trivial, if possible at all (For example, Diaspora* currently allows profile export, but not yet import into another pod).<br />
<br />
On the bright side, the Fediverse would allows any of us who are tech-savvy and dedicated enough to run our own pod and become the admins of our own data and lords of our own domain. But in reality, how many of us are really doing this?<br />
<br />
For the rest of us, what is left to do is to choose carefully which pod to join. Maybe one that is run by more than one person, a cooperative, club or association? Maybe see whether we could contribute to its operation either financially or through volunteering. And always be nice to our pod-admins, not just because they essentially own our social-media persona, but because they generally do a tedious and thankless labor of love and on top of that most likely also bear the brunt of the financial burden.<br />
<br />
While architecturally, operationally and/or organizationally distributed systems maybe interesting and may have some advantages as well as disadvantages, we should not automatically assume that they are better just because they are distributed.Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.comtag:blogger.com,1999:blog-580142149252560525.post-80488358060062872722018-12-16T16:54:00.001+01:002018-12-16T16:59:33.858+01:00Google+ Migration - Part VIII: Export to Diaspora*<span style="font-size: x-small;"><a href="https://blog.kugelfish.com/2018/12/google-migration-part-vii-conversion.html"><- Part VII: Conversion & Staging</a></span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1xPXpAzyB4eRn_L2p8NbgCe5UTiJfHWSNdq-BFJHYbwkRDcdEI1ebLN6VOsRS-H9GVn6gQS6sskZ_XwnJTXra0QFw73VE47Ln27dmcKaIXHL1j1hhWNd3qAxZgnLi08Fg7xbQhgNO1kVu/s1600/IMG_6812.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1406" data-original-width="1600" height="281" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1xPXpAzyB4eRn_L2p8NbgCe5UTiJfHWSNdq-BFJHYbwkRDcdEI1ebLN6VOsRS-H9GVn6gQS6sskZ_XwnJTXra0QFw73VE47Ln27dmcKaIXHL1j1hhWNd3qAxZgnLi08Fg7xbQhgNO1kVu/s320/IMG_6812.JPG" width="320" /></a></div>
The last stage of the process is to finally export the converted posts to Diaspora* the chosen target system. As we want these post to appear slowly and close to their original post date anniversary, this process is going to be drawn out over at least one year.<br />
<br />
While we could do this by hand, it should ideally be done by some automated process. For this to work, we need some kind of server-type machine that is up and running and connected to the Internet frequently enough during a whole year.<br />
<br />
The resource requirements are quite small, except for storing the staged data which for some users could easily be in multiple gigabytes, mostly depending on the number posts with images.<br />
<br />
Today it is quite easy to get small & cheap virtual server instances from any cloud provider, for example the micro sized compute engine instances on Google Cloud should be part of the <a href="https://cloud.google.com/compute/pricing">free tier</a> even.<br />
<br />
I also still have a few of the small, low power Rasbperry Pi boards lying around, <a href="http://blog.kugelfish.com/2012/08/kugelbot-or-what-to-do-with-raspberry-pi.html">one of which has been mirroring my public G+ posts to Twitter since 2012 and is still active today</a>.<br />
<br />
An additional challenge is that Diaspora* does at this point not offer an official and supported API. The <span style="font-family: "courier new" , "courier" , monospace;"><a href="https://github.com/marekjm/diaspy">diaspy</a></span> Python API package is essentially "screen-scraping" the callback handler URLs of the corresponding diaspora server and might break easily when the server is being upgraded to a new version, which is happening several times per year on a well maintained pod. For that reason, we are also adding additional support to send error logs including exception stack traces to an external email system so that we can hopefully notice quickly if/when something is going wrong.<br />
<br />
I am planning to run the following script about every 3 hours on my network connected Raspberry Pi server using <span style="font-family: "courier new" , "courier" , monospace;">cron</span> with the following crontab entry (see <a href="https://www.raspberrypi.org/documentation/linux/usage/cron.md">instructions for setting up a crontab entry</a>):<br />
<br />
<pre style="background-color: #eeeeee; border: 1px dashed #999999; color: black; font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><code>PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
42 */3 * * * cd /home/pi/post_bot && ./post_bot.py --staging-dir=staging --login-info=logins.json --mail
</code></pre>
<br />
This should run every <a href="https://en.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%27s_Guide_to_the_Galaxy#The_number_42">42nd</a> minute of every hour divisible by 3 on every day, assuming there is a directory at <span style="font-family: "courier new" , "courier" , monospace;">/home/pi/post_bot </span>containing the following script as <span style="font-family: "courier new" , "courier" , monospace;">post_bot.py</span>, a sub-directory staging/ with the data generated using the process described in the <a href="https://blog.kugelfish.com/2018/12/google-migration-part-vii-conversion.html">previous episode</a> and a file <span style="font-family: "courier new" , "courier" , monospace;">logins.json</span> containing the login credentials for the diaspora pod and optionally an email service to be used for error notifications.<br />
<br />
While storing passwords in clear-text on a server is a certifiably bad idea, we are at least avoiding to hard-code them in the script and storing them in a separate file instead, using JSON format, since we are already heavily using JSON for this project. The login credentials file has the following format, with the "mail" section being optional:<br />
<br />
<pre style="background-color: #eeeeee; border: 1px dashed #999999; color: black; font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><code>{
"diaspora": {
"pod-url": "<URL for diaspora pod, e.g. https://diasporing.ch>",
"username": "<username valid on this pod>",
"password": "<clear text password for diaspora pod account>"
},
"mail": {
"smtp-server": "<SMTP mail server address, e.g. mail.gmx.net>",
"username": "<username, typically email-address>",
"password": "<clear text password for email account>",
"recipient": "<recipient email address for error messages>"
}
}
</code></pre>
<br />
There are two ways to run this script: in a manual testing mode to upload a particular post, e.g. with <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">./post_bot.py --staging-dir=testing --login-info=logins.json --test=staging/20181021/20181021_4XBeoKCnV1N/</span> and the regular production mode to be called periodically, e.g. from cron e.g. as .<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">/post_bot.py --staging-dir=staging --login-info=logins.json --mail</span>. which auto-selects the next eligible post to be sent, if any.<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style><br />
<br />
For compatibility with the most recent version of diaspy, we are using python3 (e.g. install additionally with s<span style="font-family: "courier new" , "courier" , monospace;">udo apt-get install python3 python3-pip</span>) and the install the additional packages with <span style="font-family: "courier new" , "courier" , monospace;">pip3 install python-dateutil requests diaspy-api bs4</span>.<br />
<br />
However, the latest package version of diaspy is already not working properly for image upload so it may be necessary to download the latest version directly from github and copy the contents of the "<span style="font-family: "courier new" , "courier" , monospace;">diaspy</span>" subdirectory into <span style="font-family: "courier new" , "courier" , monospace;">/home/pi/post_bot</span> as a local copy of the module.<br />
<br />
As with any of the code snippets in this project, this is merely meant as an inspiration for your own implementations and not as a usable/finished product in any sense.<br />
<br />
When posting to an active social media platform, we should also be very considerate of not overwhelming the stream with archive content and be ready to engage with readers also on automatically posted content, as the goal should be to create new connections and conversations.<br />
<br />
<div>
<pre style="background-color: #eeeeee; border: 1px dashed #999999; color: black; font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><code>#!/usr/bin/env python3
import argparse
import datetime
from email.mime.text import MIMEText
import glob
from io import StringIO
import json
import logging
import logging.handlers
import os
import smtplib
import shutil
import sys
import dateutil.parser
import diaspy
ISO_DATE = '%Y%m%d'
TOOL_NAME = 'G+ post-bot'
def send_error_message(txt, email_info):
"""Send a crash/error message to a configured email address."""
server = smtplib.SMTP(email_info['smtp-server'])
server.starttls()
server.login(email_info['username'], email_info['password'])
msg = MIMEText(txt)
msg['From'] = email_info['username']
msg['To'] = email_info['recipient']
msg['Subject'] = 'error message from %s on %s' % (TOOL_NAME, os.uname()[1])
server.sendmail(email_info['username'], email_info['recipient'], msg.as_string())
server.quit()
def post_to_diaspora(post_dir, login_info):
"""Load a post from staging directory and send to diaspora server."""
cwd = os.getcwd()
os.chdir(post_dir)
content = open('content.md').read()
images = sorted(glob.glob('img_*.jpg'))
c = diaspy.connection.Connection(pod=login_info['pod-url'],
username=login_info['username'],
password=login_info['password'])
c.login()
stream = diaspy.streams.Stream(c)
if not images:
stream.post(content, provider_display_name = TOOL_NAME)
else:
ids = [stream._photoupload(name) for name in images]
stream.post(content, photos=ids, provider_display_name=TOOL_NAME)
os.chdir(cwd)
# --------------------
parser = argparse.ArgumentParser(description='Coolect post images referenced from a set of posts')
parser.add_argument('--staging-dir', dest='staging_dir', action='store', required=True)
parser.add_argument('--login-info', dest='login_info', action='store', required=True)
parser.add_argument('--test', dest='test_data', action='store')
parser.add_argument('--mail-errors', dest='mail', action='store_true')
args = parser.parse_args()
# Set up logging to both syslog and a memory buffer.
log_buffer = StringIO()
logging.basicConfig(stream=log_buffer, level=logging.INFO)
logging.getLogger().setLevel(logging.INFO)
syslog = logging.handlers.SysLogHandler(address='/dev/log')
syslog.setFormatter(logging.Formatter('diaspora-post-bot: %(levelname)s %(message)s'))
logging.getLogger().addHandler(syslog)
# Load login/authentication data from a separate file.
login_info = json.load(open(args.login_info))
if not 'diaspora' in login_info:
print('%s does not contain diaspora login section' % args.login_info)
sys.exit(1)
if args.test_data:
# Directly load a post staging directory to diaspora.
post_to_diaspora(args.test_data, login_info['diaspora'])
else:
# Find next post directory and load to diaspora.
# Intended to run un-attended from cron-job or similar at periodic intervals (e.g. every 2h)
try:
logging.info('starting export from %s' % args.staging_dir)
dirs = sorted(glob.glob(os.path.join(args.staging_dir, '[0-9]*')))
if not dirs:
logging.info('no more data to export')
sys.exit(0)
next_dir = dirs[0]
# Check if post date for next staging directory has been reached.
if dateutil.parser.parse(os.path.basename(next_dir)) > datetime.datetime.now():
logging.info('next dir not yet ready for export: %s' % os.path.basename(dirs[0]))
sys.exit(0)
logging.info('found next active staging directory %s' % next_dir)
# Find next post in staging directory or delete staging directory when empty.
posts = sorted(os.listdir(next_dir))
if not posts:
logging.info('deleting empty staging directory: %s' % next_dir)
os.rmdir(next_dir)
sys.exit(0)
# Move exported posts to a backup directory.
completion_dir = os.path.join(args.staging_dir, 'completed')
if not os.path.exists(completion_dir):
os.makedirs(completion_dir)
# Send next post to diaspora server.
post_dir = os.path.join(next_dir, posts[0])
logging.info('posting %s...' % post_dir)
post_to_diaspora(post_dir, login_info['diaspora'])
shutil.move(post_dir, completion_dir)
logging.info('post completed')
sys.exit(0)
except (KeyboardInterrupt, SystemExit):
sys.exit(1)
except Exception as e:
logging.exception('error in main loop')
if args.mail and 'mail' in login_info:
send_error_message(log_buffer.getvalue(), login_info['mail'])
sys.exit(1)
</code></pre>
<br /></div>
<div>
<br /></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>Bernhard Suterhttp://www.blogger.com/profile/15395349968095219960noreply@blogger.com