<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Blogs on Byteli</title><link>https://www.byteli.com/blog/</link><description>Recent content in Blogs on Byteli</description><generator>Hugo -- gohugo.io</generator><language>en-US</language><managingEditor>binli@byteli.com (Bin Li)</managingEditor><webMaster>binli@byteli.com (Bin Li)</webMaster><copyright>Byteli (CC BY-NC 4.0)</copyright><lastBuildDate>Fri, 21 Mar 2025 10:50:47 +0000</lastBuildDate><atom:link href="https://www.byteli.com/blog/index.xml" rel="self" type="application/rss+xml"/><item><title>Fetching and Visualizing Forex Data in Python with yfinance</title><link>https://www.byteli.com/blog/2025/fetching_and_visualizing_forex_data_in_python_with_yfinance/</link><pubDate>Fri, 21 Mar 2025 10:50:47 +0000</pubDate><author>binli@byteli.com (Bin Li)</author><guid>https://www.byteli.com/blog/2025/fetching_and_visualizing_forex_data_in_python_with_yfinance/</guid><description>&lt;p>I am preparing for a Forex-related interview, so after some research, I’ve summarized how to get Forex data in Python. In this blog, we will use &lt;code>yfinance&lt;/code> and &lt;code>seaborn&lt;/code> to fetch hourly exchange rates for &lt;strong>EUR/USD&lt;/strong>, check the data quality, and finally, plot the closing prices in a line plot.&lt;/p>
&lt;h2 id="install-packages">Install Packages&lt;/h2>
&lt;p>To get started, we need the following packages:&lt;/p>
&lt;ul>
&lt;li>&lt;a href="https://github.com/ranaroussi/yfinance">yfinance&lt;/a>: to fetch Forex data&lt;/li>
&lt;li>&lt;a href="https://matplotlib.org/">matplotlib&lt;/a>: for plotting&lt;/li>
&lt;li>&lt;a href="https://seaborn.pydata.org/">seaborn&lt;/a>: for enhanced plotting with better visualization&lt;/li>
&lt;/ul>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">pip install yfinance matplotlib seaborn&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;h2 id="fetch-the-data">Fetch the Data&lt;/h2>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-python" data-lang="python">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">&lt;span class="kn">import&lt;/span> &lt;span class="nn">yfinance&lt;/span> &lt;span class="k">as&lt;/span> &lt;span class="nn">yf&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">&lt;span class="kn">import&lt;/span> &lt;span class="nn">seaborn&lt;/span> &lt;span class="k">as&lt;/span> &lt;span class="nn">sns&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl">&lt;span class="kn">import&lt;/span> &lt;span class="nn">matplotlib.pyplot&lt;/span> &lt;span class="k">as&lt;/span> &lt;span class="nn">plt&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">&lt;span class="n">sns&lt;/span>&lt;span class="o">.&lt;/span>&lt;span class="n">set_style&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="s2">&amp;#34;whitegrid&amp;#34;&lt;/span>&lt;span class="p">)&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="c1"># Define the forex pair and date range&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">&lt;span class="n">pair&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;EURUSD=X&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="c1"># Define the start date and end date&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl">&lt;span class="n">start_date&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;2024-01-01&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl">&lt;span class="n">end_date&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;2024-12-31&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl">&lt;span class="c1"># Define the interval&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl">&lt;span class="n">interval&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s1">&amp;#39;1h&amp;#39;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl">&lt;span class="c1"># Fetch forex data&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl">&lt;span class="n">forex_data&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="n">yf&lt;/span>&lt;span class="o">.&lt;/span>&lt;span class="n">Ticker&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="n">pair&lt;/span>&lt;span class="p">)&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">17&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">18&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">19&lt;/span>&lt;span class="cl">&lt;span class="c1"># Fetch historical forex data&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">20&lt;/span>&lt;span class="cl">&lt;span class="n">data&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="n">forex_data&lt;/span>&lt;span class="o">.&lt;/span>&lt;span class="n">history&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="n">start&lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="n">start_date&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="n">end&lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="n">end_date&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="n">interval&lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="n">interval&lt;/span>&lt;span class="p">)&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">21&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">22&lt;/span>&lt;span class="cl">&lt;span class="c1"># Data overview&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">23&lt;/span>&lt;span class="cl">&lt;span class="n">data&lt;/span>&lt;span class="o">.&lt;/span>&lt;span class="n">info&lt;/span>&lt;span class="p">()&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">24&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">25&lt;/span>&lt;span class="cl">&lt;span class="s2">&amp;#34;&amp;#34;&amp;#34;Output
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">26&lt;/span>&lt;span class="cl">&lt;span class="s2">&amp;lt;class &amp;#39;pandas.core.frame.DataFrame&amp;#39;&amp;gt;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">27&lt;/span>&lt;span class="cl">&lt;span class="s2">DatetimeIndex: 6178 entries, 2024-01-01 18:00:00+00:00 to 2024-12-30 23:00:00+00:00
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">28&lt;/span>&lt;span class="cl">&lt;span class="s2">Data columns (total 7 columns):
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">29&lt;/span>&lt;span class="cl">&lt;span class="s2"> # Column Non-Null Count Dtype
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">30&lt;/span>&lt;span class="cl">&lt;span class="s2">--- ------ -------------- -----
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">31&lt;/span>&lt;span class="cl">&lt;span class="s2"> 0 Open 6178 non-null float64
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">32&lt;/span>&lt;span class="cl">&lt;span class="s2"> 1 High 6178 non-null float64
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">33&lt;/span>&lt;span class="cl">&lt;span class="s2"> 2 Low 6178 non-null float64
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">34&lt;/span>&lt;span class="cl">&lt;span class="s2"> 3 Close 6178 non-null float64
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">35&lt;/span>&lt;span class="cl">&lt;span class="s2"> 4 Volume 6178 non-null int64
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">36&lt;/span>&lt;span class="cl">&lt;span class="s2"> 5 Dividends 6178 non-null float64
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">37&lt;/span>&lt;span class="cl">&lt;span class="s2"> 6 Stock Splits 6178 non-null float64
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">38&lt;/span>&lt;span class="cl">&lt;span class="s2">dtypes: float64(6), int64(1)
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">39&lt;/span>&lt;span class="cl">&lt;span class="s2">&amp;#34;&amp;#34;&amp;#34;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;h2 id="data-quality-check">Data Quality Check&lt;/h2>
&lt;h3 id="missing-values">Missing values&lt;/h3>
&lt;p>We can check the missing values by&lt;/p></description><content:encoded><![CDATA[<p>I am preparing for a Forex-related interview, so after some research, I’ve summarized how to get Forex data in Python. In this blog, we will use <code>yfinance</code> and <code>seaborn</code> to fetch hourly exchange rates for <strong>EUR/USD</strong>, check the data quality, and finally, plot the closing prices in a line plot.</p>
<h2 id="install-packages">Install Packages</h2>
<p>To get started, we need the following packages:</p>
<ul>
<li><a href="https://github.com/ranaroussi/yfinance">yfinance</a>: to fetch Forex data</li>
<li><a href="https://matplotlib.org/">matplotlib</a>: for plotting</li>
<li><a href="https://seaborn.pydata.org/">seaborn</a>: for enhanced plotting with better visualization</li>
</ul>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">pip install yfinance matplotlib seaborn</span></span></code></pre></div><h2 id="fetch-the-data">Fetch the Data</h2>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="kn">import</span> <span class="nn">yfinance</span> <span class="k">as</span> <span class="nn">yf</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="kn">import</span> <span class="nn">seaborn</span> <span class="k">as</span> <span class="nn">sns</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">set_style</span><span class="p">(</span><span class="s2">&#34;whitegrid&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="c1"># Define the forex pair and date range</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="n">pair</span> <span class="o">=</span> <span class="s2">&#34;EURUSD=X&#34;</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="c1"># Define the start date and end date</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="n">start_date</span> <span class="o">=</span> <span class="s2">&#34;2024-01-01&#34;</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="n">end_date</span> <span class="o">=</span> <span class="s2">&#34;2024-12-31&#34;</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="c1"># Define the interval</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="n">interval</span> <span class="o">=</span> <span class="s1">&#39;1h&#39;</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl">
</span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="c1"># Fetch forex data</span>
</span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="n">forex_data</span> <span class="o">=</span> <span class="n">yf</span><span class="o">.</span><span class="n">Ticker</span><span class="p">(</span><span class="n">pair</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">17</span><span class="cl">
</span></span><span class="line"><span class="ln">18</span><span class="cl">
</span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="c1"># Fetch historical forex data</span>
</span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="n">data</span> <span class="o">=</span> <span class="n">forex_data</span><span class="o">.</span><span class="n">history</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">start_date</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="n">end_date</span><span class="p">,</span> <span class="n">interval</span><span class="o">=</span><span class="n">interval</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">21</span><span class="cl">
</span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="c1"># Data overview</span>
</span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="n">data</span><span class="o">.</span><span class="n">info</span><span class="p">()</span>
</span></span><span class="line"><span class="ln">24</span><span class="cl">
</span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="s2">&#34;&#34;&#34;Output
</span></span></span><span class="line"><span class="ln">26</span><span class="cl"><span class="s2">&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;
</span></span></span><span class="line"><span class="ln">27</span><span class="cl"><span class="s2">DatetimeIndex: 6178 entries, 2024-01-01 18:00:00+00:00 to 2024-12-30 23:00:00+00:00
</span></span></span><span class="line"><span class="ln">28</span><span class="cl"><span class="s2">Data columns (total 7 columns):
</span></span></span><span class="line"><span class="ln">29</span><span class="cl"><span class="s2"> #   Column        Non-Null Count  Dtype
</span></span></span><span class="line"><span class="ln">30</span><span class="cl"><span class="s2">---  ------        --------------  -----
</span></span></span><span class="line"><span class="ln">31</span><span class="cl"><span class="s2"> 0   Open          6178 non-null   float64
</span></span></span><span class="line"><span class="ln">32</span><span class="cl"><span class="s2"> 1   High          6178 non-null   float64
</span></span></span><span class="line"><span class="ln">33</span><span class="cl"><span class="s2"> 2   Low           6178 non-null   float64
</span></span></span><span class="line"><span class="ln">34</span><span class="cl"><span class="s2"> 3   Close         6178 non-null   float64
</span></span></span><span class="line"><span class="ln">35</span><span class="cl"><span class="s2"> 4   Volume        6178 non-null   int64
</span></span></span><span class="line"><span class="ln">36</span><span class="cl"><span class="s2"> 5   Dividends     6178 non-null   float64
</span></span></span><span class="line"><span class="ln">37</span><span class="cl"><span class="s2"> 6   Stock Splits  6178 non-null   float64
</span></span></span><span class="line"><span class="ln">38</span><span class="cl"><span class="s2">dtypes: float64(6), int64(1)
</span></span></span><span class="line"><span class="ln">39</span><span class="cl"><span class="s2">&#34;&#34;&#34;</span></span></span></code></pre></div><h2 id="data-quality-check">Data Quality Check</h2>
<h3 id="missing-values">Missing values</h3>
<p>We can check the missing values by</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="n">data</span><span class="o">.</span><span class="n">isna</span><span class="p">()</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl">
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="s2">&#34;&#34;&#34; Output
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s2">Open            0
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s2">High            0
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s2">Low             0
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s2">Close           0
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s2">Volume          0
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s2">Dividends       0
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s2">Stock Splits    0
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="s2">dtype: int64
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="s2">&#34;&#34;&#34;</span></span></span></code></pre></div><p>No missing values, nice :)</p>
<h3 id="removing-irrelevant-columns">Removing Irrelevant Columns</h3>
<p>Upon closer inspection, we notice that the <code>Volume</code>, <code>Dividends</code>, and <code>Stock Splits</code> columns have all values equal to 0. These columns are not useful for our analysis, so let&rsquo;s drop them.</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="n">cols</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;Volume&#39;</span><span class="p">,</span> <span class="s1">&#39;Dividends&#39;</span><span class="p">,</span><span class="s1">&#39;Stock Splits&#39;</span><span class="p">]</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="n">cols</span><span class="p">]</span> <span class="o">==</span> <span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">all</span><span class="p">()</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">
</span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s2">&#34;&#34;&#34; Output
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s2">Volume          True
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s2">Dividends       True
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s2">Stock Splits    True
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s2">dtype: bool
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s2">&#34;&#34;&#34;</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl">
</span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="n">data</span><span class="o">.</span><span class="n">drop</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span> <span class="n">cols</span><span class="p">,</span> <span class="n">inplace</span> <span class="o">=</span> <span class="kc">True</span><span class="p">)</span></span></span></code></pre></div><p>After this, we have a cleaned DataFrame.</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="n">data</span><span class="o">.</span><span class="n">info</span><span class="p">()</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl">
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="s2">&#34;&#34;&#34; Output
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s2">DatetimeIndex: 6178 entries, 2024-01-01 18:00:00+00:00 to 2024-12-30 23:00:00+00:00
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s2">Data columns (total 7 columns):
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s2"> #   Column        Non-Null Count  Dtype
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s2">---  ------        --------------  -----
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s2"> 0   Open          6178 non-null   float64
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s2"> 1   High          6178 non-null   float64
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s2"> 2   Low           6178 non-null   float64
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="s2"> 3   Close         6178 non-null   float64
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="s2"> 4   Volume        6178 non-null   int64
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="s2"> 5   Dividends     6178 non-null   float64
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="s2"> 6   Stock Splits  6178 non-null   float64
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="s2">dtypes: float64(6), int64(1)
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="s2">&#34;&#34;&#34;</span></span></span></code></pre></div><h2 id="data-visulazation">Data Visulazation</h2>
<p>We will now use <code>seaborn</code> to plot the hourly closing prices.</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="kn">import</span> <span class="nn">seaborn</span> <span class="k">as</span> <span class="nn">sns</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">
</span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="c1"># Set up the plot style</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">set_style</span><span class="p">(</span><span class="s2">&#34;whitegrid&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">))</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl">
</span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="c1"># Plot the hourly closing prices</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">lineplot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">data</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="n">data</span><span class="p">[</span><span class="s2">&#34;Close&#34;</span><span class="p">])</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl">
</span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="c1"># Formatting the plot</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s2">&#34;Date&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s2">&#34;EUR/USD Hourly Closing Price&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">&#34;EUR/USD Hourly Closing Price (2024)&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">xticks</span><span class="p">(</span><span class="n">rotation</span><span class="o">=</span><span class="mi">45</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span></span></span></code></pre></div><p>This will generate a line plot showing the hourly closing prices for the EUR/USD pair in 2024.</p>
<p><img
src={forex_plot.src}
alt='EUR/USD Hourly Closing Price (2024)'
className='rounded-lg hover:scale-[1.01] drop-shadow-xl'
/></p>
<p><img src="/blog/2025/fetching_and_visualizing_forex_data_in_python_with_yfinance/EUR_USD_Hourly_Closing_Price_2024.png" alt="EUR_USD_Hourly_Closing_Price_2024"></p>
<h2 id="conclusion">Conclusion</h2>
<p>In this blog, we learned how to fetch Forex data using the <code>yfinance</code> library, cleaned the dataset by removing irrelevant columns, and visualized the EUR/USD hourly closing prices with <code>seaborn</code>. This simple step can serve as the foundation for more complex Forex analysis and hedging strategies.</p>
]]></content:encoded></item><item><title>Why I Built a Home Lab</title><link>https://www.byteli.com/blog/2025/why_i_built_a_home_lab/</link><pubDate>Sat, 18 Jan 2025 22:58:58 +0000</pubDate><author>binli@byteli.com (Bin Li)</author><guid>https://www.byteli.com/blog/2025/why_i_built_a_home_lab/</guid><description>&lt;h2 id="building-for-the-long-term">Building for the Long Term&lt;/h2>
&lt;p>I see this home lab as the start of a long-term (10+ years) project—one that allows me to enjoy the process of building and creating. Technology is a vast and dynamic field, with new developments emerging every second: new languages (like Rust), frameworks (React 19), tools (Heptabase), and AIs (DeepSeek). While fascinating, it’s also overwhelming. I often find myself hoarding tutorials, trapped in what many call &amp;ldquo;tutorial hell&amp;rdquo;—living under the illusion of learning while simply saving resources I never have time to explore.&lt;/p></description><content:encoded><![CDATA[<h2 id="building-for-the-long-term">Building for the Long Term</h2>
<p>I see this home lab as the start of a long-term (10+ years) project—one that allows me to enjoy the process of building and creating. Technology is a vast and dynamic field, with new developments emerging every second: new languages (like Rust), frameworks (React 19), tools (Heptabase), and AIs (DeepSeek). While fascinating, it’s also overwhelming. I often find myself hoarding tutorials, trapped in what many call &ldquo;tutorial hell&rdquo;—living under the illusion of learning while simply saving resources I never have time to explore.</p>
<p>It’s time to break free from that cycle. I want to shift to a project-based, interest-driven approach. I don’t know exactly where this journey will take me, but I’m certain it will be anything but boring. Plus, creating tangible improvements in my daily life will bring a much-needed sense of achievement.</p>
<h2 id="control-the-system-not-the-other-way-around">Control the System, Not the Other Way Around</h2>
<p>Linux has always fascinated me with its elegance, and I’ve long wanted to explore it in depth. With academic pressures now behind me, this feels like the perfect time to dive into this mysterious yet exciting world and experiment with cutting-edge technologies. My trusty Raspberry Pi 4 Model B remains an excellent tool for this exploration.</p>
<p>Five years ago, my first experience with CentOS was both thrilling and intimidating. The freedom to control everything felt powerful but also risky—one careless command could ruin the entire system. Yet, that same freedom gave me an exhilarating sense of ownership. For the first time, I had full control over both the operating system and the hardware beneath it. The possibilities felt limitless, and the sense of empowerment was incredible.</p>
<p><img src="https://images7.memedroid.com/images/UPLOADED540/6583eafbb994b.jpeg" alt="Linux and Windows meme"></p>
<h2 id="reclaiming-my-attention">Reclaiming My Attention</h2>
<p>Last year, I struggled with information overload—too much content demanding too little focus. To regain my concentration, I intentionally quit most social media platforms. It became clear to me that attention is my most valuable resource. During this process, I rediscovered the beauty of old-school RSS. It’s exactly what I need: a way to read content with depth and authenticity, written by people who aim to express, not impress.</p>
<p>Since I enjoy reading on my Kindle, I was thrilled to discover the potential of a self-hosted send-to-Kindle service like <a href="https://github.com/facundoolano/feedi">Feedi</a>. I deeply resonated with the creator(Facundo Olano)&rsquo;s motivation on the blog <a href="https://olano.dev/blog/reclaiming-the-web-with-a-personal-reader/">Reclaiming the Web with a Personal Reader</a>:</p>
<blockquote>
<p><em>It felt refreshing to be back in control of my feed, to receive strictly chronological updates instead of having an algorithmic middleman trying to sell me stuff.</em></p></blockquote>
<p>This inspired me to set up a home server or lab to fully take control of my digital consumption.</p>
<h2 id="looking-forward">Looking Forward</h2>
<p>In the future, I hope it becomes a great space for me to experiment with and configure different things that appeal to me, fueling my curiosity even further. But most importantly—it’s all about having fun.</p>
]]></content:encoded></item><item><title>How to Fix the f-string Error: Unmatched '(' in yfinance</title><link>https://www.byteli.com/blog/2024/fix_yfinance_error/</link><pubDate>Mon, 21 Oct 2024 00:00:00 +0000</pubDate><author>binli@byteli.com (Bin Li)</author><guid>https://www.byteli.com/blog/2024/fix_yfinance_error/</guid><description>&lt;p>If you&amp;rsquo;re using the latest version of the yfinance library (version &lt;code>0.2.45&lt;/code> as of now), you might encounter the following error:&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">Traceback &lt;span class="o">(&lt;/span>most recent call last&lt;span class="o">)&lt;/span>:
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl"> File ~/anaconda3/envs/fin/lib/python3.10/site-packages/IPython/core/interactiveshell.py:3577 in run_code
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl"> exec&lt;span class="o">(&lt;/span>code_obj, self.user_global_ns, self.user_ns&lt;span class="o">)&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl"> Cell In&lt;span class="o">[&lt;/span>1&lt;span class="o">]&lt;/span>, line &lt;span class="m">1&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl"> import yfinance as yf
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl"> File ~/anaconda3/envs/fin/lib/python3.10/site-packages/yfinance/__init__.py:30
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl"> from .screener.screener import Screener
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl"> File ~/anaconda3/envs/fin/lib/python3.10/site-packages/yfinance/screener/__init__.py:1
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl"> from .screener import Screener
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl"> File ~/anaconda3/envs/fin/lib/python3.10/site-packages/yfinance/screener/screener.py:101
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl"> logger.error&lt;span class="o">(&lt;/span>f&lt;span class="s2">&amp;#34;Failed to get screener data for &amp;#39;{self._body.get(&amp;#39;query&amp;#39;, &amp;#34;&lt;/span>query not set&lt;span class="s2">&amp;#34;)}&amp;#39; reason: {e}&amp;#34;&lt;/span>&lt;span class="o">)&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">17&lt;/span>&lt;span class="cl"> ^
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">18&lt;/span>&lt;span class="cl">SyntaxError: f-string: unmatched &lt;span class="s1">&amp;#39;(&amp;#39;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>&lt;img src="https://www.byteli.com/blog/2024/fix_yfinance_error/yfinance_error.png" alt="Error in yfinance">&lt;/p></description><content:encoded><![CDATA[<p>If you&rsquo;re using the latest version of the yfinance library (version <code>0.2.45</code> as of now), you might encounter the following error:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln"> 1</span><span class="cl">Traceback <span class="o">(</span>most recent call last<span class="o">)</span>:
</span></span><span class="line"><span class="ln"> 2</span><span class="cl">
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">  File ~/anaconda3/envs/fin/lib/python3.10/site-packages/IPython/core/interactiveshell.py:3577 in run_code
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">    exec<span class="o">(</span>code_obj, self.user_global_ns, self.user_ns<span class="o">)</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl">
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">  Cell In<span class="o">[</span>1<span class="o">]</span>, line <span class="m">1</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl">    import yfinance as yf
</span></span><span class="line"><span class="ln"> 8</span><span class="cl">
</span></span><span class="line"><span class="ln"> 9</span><span class="cl">  File ~/anaconda3/envs/fin/lib/python3.10/site-packages/yfinance/__init__.py:30
</span></span><span class="line"><span class="ln">10</span><span class="cl">    from .screener.screener import Screener
</span></span><span class="line"><span class="ln">11</span><span class="cl">
</span></span><span class="line"><span class="ln">12</span><span class="cl">  File ~/anaconda3/envs/fin/lib/python3.10/site-packages/yfinance/screener/__init__.py:1
</span></span><span class="line"><span class="ln">13</span><span class="cl">    from .screener import Screener
</span></span><span class="line"><span class="ln">14</span><span class="cl">
</span></span><span class="line"><span class="ln">15</span><span class="cl">  File ~/anaconda3/envs/fin/lib/python3.10/site-packages/yfinance/screener/screener.py:101
</span></span><span class="line"><span class="ln">16</span><span class="cl">    logger.error<span class="o">(</span>f<span class="s2">&#34;Failed to get screener data for &#39;{self._body.get(&#39;query&#39;, &#34;</span>query not set<span class="s2">&#34;)}&#39; reason: {e}&#34;</span><span class="o">)</span>
</span></span><span class="line"><span class="ln">17</span><span class="cl">                                                                              ^
</span></span><span class="line"><span class="ln">18</span><span class="cl">SyntaxError: f-string: unmatched <span class="s1">&#39;(&#39;</span></span></span></code></pre></div><p><img src="/blog/2024/fix_yfinance_error/yfinance_error.png" alt="Error in yfinance"></p>
<p>Don&rsquo;t worry! This error is a known issue with the yfinance <code>0.2.45</code> source code, and it&rsquo;s easy to fix. Below, I’ll explain what causes this error and two simple ways to resolve it.</p>
<h2 id="why-this-happens">Why This Happens</h2>
<p>The issue occurs in this line of the <a href="https://github.com/ranaroussi/yfinance/blob/main/yfinance/screener/screener.py#L101">yfinance source code</a>:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-py" data-lang="py"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;Failed to get screener data for &#39;</span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">_body</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;query&#39;</span><span class="p">,</span> <span class="s2">&#34;query not set&#34;</span><span class="p">)</span><span class="si">}</span><span class="s2">&#39; reason: </span><span class="si">{</span><span class="n">e</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span></span></span></code></pre></div><p>The problem lies in the mixing of single and double quotes inside the <code>.get()</code> method. Python tries to process both <code>'query'</code> and <code>&quot;query not set&quot;</code> within the same string, causing confusion. Since the string started with a single quote, Python expects a matching single quote but encounters a double quote, leading to a <strong>syntax error</strong>.</p>
<h2 id="two-simple-fixes">Two Simple Fixes</h2>
<p>You can fix this issue in one of two ways: either by downgrading to an earlier version of yfinance or by manually correcting the source code. Here’s how:</p>
<h3 id="1-downgrade-to-a-previous-yfinance-version">1. Downgrade to a Previous yfinance Version</h3>
<p>If you don’t need the latest features of yfinance <code>0.2.45</code>, simply downgrade to version <code>0.2.44</code> by running this command in your terminal:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">pip install --force-reinstall -v <span class="s2">&#34;yfinance==0.2.44&#34;</span></span></span></code></pre></div><p>After downgrading, restart your Jupyter notebook, and the issue should be resolved.</p>
<h3 id="2-manually-edit-the-source-code">2. Manually Edit the Source Code</h3>
<p>If you prefer to stick with yfinance <code>0.2.45</code> for its new features, you can manually correct the problematic line in the source code.</p>
<p>Follow these steps:</p>
<ol>
<li>Locate the file: <code>yfinance/screener/screener.py</code>.</li>
<li>Find the offending line:</li>
</ol>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">logger.error<span class="o">(</span>f<span class="s2">&#34;Failed to get screener data for &#39;{self._body.get(&#39;query&#39;, &#34;</span>query not set<span class="s2">&#34;)}&#39; reason: {e}&#34;</span><span class="o">)</span></span></span></code></pre></div><p>and change it to</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">logger.error<span class="o">(</span>f<span class="s2">&#34;Failed to get screener data for &#39;{self._body.get(&#39;query&#39;, &#39;query not set&#39;)}&#39; reason: {e}&#34;</span><span class="o">)</span></span></span></code></pre></div><p>This small change ensures consistent use of single quotes, preventing the syntax error.</p>
<p><img src="/blog/2024/fix_yfinance_error/yfinance_solved.png" alt="yfinance works now 😄"></p>
<h2 id="conclusion">Conclusion</h2>
<p>This error in yfinance version <code>0.2.45</code> is easily fixable, either by downgrading to a previous version or manually editing the source code. Choose the method that works best for your needs, and you’ll be back to analyzing financial data in no time!</p>
]]></content:encoded></item><item><title>SEO Express: Get Your Website Google-Indexed in 48 Hours</title><link>https://www.byteli.com/blog/2024/seo_script/</link><pubDate>Mon, 05 Feb 2024 21:20:10 +0000</pubDate><author>binli@byteli.com (Bin Li)</author><guid>https://www.byteli.com/blog/2024/seo_script/</guid><description>&lt;p>Ever since I launched my blog, I have often encountered the same indexing problem with Google - it takes far too long for my newly published posts to be indexed, and I have to manually submit them using a URL inspection tool - which is tedious and annoying. Until recently, I found this &lt;a href="https://github.com/goenning/google-indexing-script">Google Indexing Script&lt;/a> on GitHub and use this script to submit new links to Google and get them indexed, and the problem has just been solved.&lt;/p></description><content:encoded><![CDATA[<p>Ever since I launched my blog, I have often encountered the same indexing problem with Google - it takes far too long for my newly published posts to be indexed, and I have to manually submit them using a URL inspection tool - which is tedious and annoying. Until recently, I found this <a href="https://github.com/goenning/google-indexing-script">Google Indexing Script</a> on GitHub and use this script to submit new links to Google and get them indexed, and the problem has just been solved.</p>
<p>In this blog, I will walk you through all the technical settings of this lightweight yet powerful tool and end up getting your site indexed in Google within 48 hours.</p>
<h2 id="-google-account-preparation">🔗 Google Account Preparation</h2>
<p>Register a <a href="https://console.cloud.google.com/">Google Cloud</a> account</p>
<p>Enable Google Search API <a href="https://console.cloud.google.com/marketplace/product/google/searchconsole.googleapis.com?q=search&amp;referrer=search&amp;project=red-button-335005">here</a>.</p>
<p>Enable Web Search Indexing API <a href="https://console.cloud.google.com/marketplace/product/google/indexing.googleapis.com?q=search&amp;referrer=search&amp;project=red-button-335005">here</a>.</p>
<h3 id="create-a-service-account">Create a service account</h3>
<p>Create a service account <a href="https://console.cloud.google.com/iam-admin/serviceaccounts?project=red-button-335005">here</a></p>
<p><img src="/blog/2024/seo_script/create_a_service_account.png" alt="Create a service account"></p>
<p>Then click the service account we just created, and go to the <code>Keys</code> panel to create a new key.</p>
<p><img src="/blog/2024/seo_script/generate_key_2.png" alt="Generate a key"></p>
<p><img src="/blog/2024/seo_script/download_key.png" alt="Download the key"></p>
<p>Then a window will pop up and we just download the key we generated; Then, we change its name to <code>service_account.json</code>.</p>
<p><img src="/blog/2024/seo_script/download_key_2.png" alt="Generate a key"></p>
<h3 id="add-your-service-account-to-google-console-as-owner">Add your service account to Google console as Owner</h3>
<p>Copy our service account email to clipboard, we will use this in a moment.</p>
<p><img src="/blog/2024/seo_script/create_owner.png" alt="Create a owner user"></p>
<p>Visit Google Console. In &lsquo;Settings&rsquo; panel, click &lsquo;add user&rsquo;. Then, in the pop-up window, paste our service account email and make sure the permission is set to &lsquo;Owner&rsquo;</p>
<p><img src="/blog/2024/seo_script/create_owner_2.png" alt="Create a owner user"></p>
<p><img src="/blog/2024/seo_script/create_owner_3.png" alt="Create a owner user"></p>
<p>After clicking &lsquo;Add&rsquo;, you service account should be successfully added to Google Console:</p>
<p><img src="/blog/2024/seo_script/create_owner_4.png" alt="Create a owner user"></p>
<h2 id="-environment-preparation">🛠️ Environment Preparation</h2>
<h3 id="nodejs">Node.js</h3>
<p>Go to <a href="https://nodejs.org/en/download/">Node.js</a> , download and install the Latest LTS Version of Node.js, it&rsquo;s <code>20.11.0</code> at current time.</p>
<p>Open the terminal of your computer, check whether Node.js has successfully installed</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="ln">1</span><span class="cl">node -v
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="c1"># 20.11.0</span></span></span></code></pre></div><h3 id="git">Git</h3>
<p>You can check whether you have installed <code>Git</code> by running this command in the terminal</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="ln">1</span><span class="cl">git version
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="c1"># git version 2.33.0</span></span></span></code></pre></div><p>If you haven&rsquo;t installed it, you can refer to <a href="https://github.com/git-guides/install-git">this guide</a>.</p>
<h2 id="-script-configuration">✏️ Script Configuration</h2>
<p>This is the last step in our progress.</p>
<h3 id="download-the-script">download the script</h3>
<p>use this command to download the script into our local computer:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="ln">1</span><span class="cl">git clone https://github.com/goenning/google-indexing-script.git</span></span></code></pre></div><h3 id="install-dependencies">install dependencies</h3>
<p>Open the folder we just downloaded in an IDE; I personally use &lsquo;WebStorm&rsquo;. Then, in the terminal, run the following command to install all the necessary dependencies of this script.</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="ln">1</span><span class="cl">npm install</span></span></code></pre></div><h3 id="set-the-credentials">set the credentials</h3>
<p>Move the <code>service_account.json</code> we downloaded in previous step to the same same folder as the script.</p>
<p>The directory should be the following:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"><span class="line"><span class="ln">1</span><span class="cl">.
</span></span><span class="line"><span class="ln">2</span><span class="cl">├── LICENSE
</span></span><span class="line"><span class="ln">3</span><span class="cl">├── README.md
</span></span><span class="line"><span class="ln">4</span><span class="cl">├── output.png
</span></span><span class="line"><span class="ln">5</span><span class="cl">├── package-lock.json
</span></span><span class="line"><span class="ln">6</span><span class="cl">├── package.json
</span></span><span class="line"><span class="ln">7</span><span class="cl">├── src
</span></span><span class="line"><span class="ln">8</span><span class="cl">├── node_modules
</span></span><span class="line"><span class="ln">9</span><span class="cl">└── service_account.json</span></span></code></pre></div><h3 id="run-the-script">Run the Script</h3>
<p>In the terminal, run</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="ln">1</span><span class="cl">npm run index yourWebsiteDomain.com
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="c1"># OR</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">npm run index https://yourWebsiteDomain.com</span></span></code></pre></div><p>For instance, since the domain of my blog is &lsquo;byteli&rsquo;, so I just run</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="ln">1</span><span class="cl">npm run index byteli.com</span></span></code></pre></div><p>And we just submitted the unindexed pages in seconds 🎉</p>
<p><img src="/blog/2024/seo_script/submit_pages.png" alt="Submit pages"></p>
<h3 id="check-the-updates">Check the Updates</h3>
<p>In one or two days, visit <a href="https://search.google.com/search-console/about">Google Search Console</a>, under &lsquo;Indexing&rsquo; - &lsquo;Pages&rsquo;, you can view the updates of the newly indexed pages.</p>
<h2 id="-conclusion">🎉 Conclusion</h2>
<p>In conclusion, overcoming the frustration of delayed Google indexing is a game-changer for any blogger. The journey outlined in this blog, from Google account preparation to environment setup and script configuration, provides a comprehensive guide to tackle the indexing challenge effectively. As you embark on this journey, embrace the power of automation and witness the positive impact on your blog&rsquo;s discoverability. Happy blogging!</p>
]]></content:encoded></item><item><title>Data Visualization: `hue` in Seaborn</title><link>https://www.byteli.com/blog/2024/hue-in-seaborn/</link><pubDate>Thu, 25 Jan 2024 23:46:27 +0000</pubDate><author>binli@byteli.com (Bin Li)</author><guid>https://www.byteli.com/blog/2024/hue-in-seaborn/</guid><description>&lt;p>Photo by &lt;a href="https://unsplash.com/@ro_ka?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Robert Katzki&lt;/a> on &lt;a href="https://unsplash.com/photos/multicolored-wall-in-shallow-focus-photography-jbtfM0XBeRc?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash&lt;/a>&lt;/p>
&lt;p>The &lt;code>hue&lt;/code> parameter in &lt;a href="https://seaborn.pydata.org/">Seaborn&lt;/a> allows for the seamless integration of categorical variables, introducing a spectrum of colors that not only enhances the aesthetic appeal of your plots but also provides a powerful tool for conveying intricate relationships within your data. In this blog, we will embark on a journey to explore the nuances of using &amp;lsquo;hue&amp;rsquo; in Seaborn, unlocking its potential to reveal insights, distinguish patterns, and elevate the clarity of your visual narratives.&lt;/p></description><content:encoded><![CDATA[<p>Photo by <a href="https://unsplash.com/@ro_ka?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Robert Katzki</a> on <a href="https://unsplash.com/photos/multicolored-wall-in-shallow-focus-photography-jbtfM0XBeRc?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a></p>
<p>The <code>hue</code> parameter in <a href="https://seaborn.pydata.org/">Seaborn</a> allows for the seamless integration of categorical variables, introducing a spectrum of colors that not only enhances the aesthetic appeal of your plots but also provides a powerful tool for conveying intricate relationships within your data. In this blog, we will embark on a journey to explore the nuances of using &lsquo;hue&rsquo; in Seaborn, unlocking its potential to reveal insights, distinguish patterns, and elevate the clarity of your visual narratives.</p>
<h2 id="-usage">📊 Usage</h2>
<p>Firstly, We will load <code>tips</code> dataset from the online repository in Seaborn. This dataset contains the following data:</p>
<blockquote>
<p>One waiter recorded information about each tip he received over a period of a few months working in one restaurant</p></blockquote>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln">1</span><span class="cl"><span class="kn">import</span> <span class="nn">seaborn</span> <span class="k">as</span> <span class="nn">sns</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">
</span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="c1"># Load sample data</span>
</span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="n">data</span> <span class="o">=</span> <span class="n">sns</span><span class="o">.</span><span class="n">load_dataset</span><span class="p">(</span><span class="s2">&#34;tips&#34;</span><span class="p">)</span></span></span></code></pre></div><p>Like always, We will inspect our data then:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="n">data</span><span class="o">.</span><span class="n">info</span><span class="p">()</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl">
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="s2">&#34;&#34;&#34;
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s2">&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s2">RangeIndex: 244 entries, 0 to 243
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s2">Data columns (total 7 columns):
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s2"> #   Column      Non-Null Count  Dtype
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s2">-- -----     ------------- -----
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s2"> 0   total_bill  244 non-null    float64
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s2"> 1   tip         244 non-null    float64
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="s2"> 2   sex         244 non-null    category
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="s2"> 3   smoker      244 non-null    category
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="s2"> 4   day         244 non-null    category
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="s2"> 5   time        244 non-null    category
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="s2"> 6   size        244 non-null    int64
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="s2">dtypes: category(4), float64(2), int64(1)
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="s2">memory usage: 7.4 KB
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="s2">&#34;&#34;&#34;</span></span></span></code></pre></div><p>According to the <a href="https://rdrr.io/cran/reshape2/man/tips.html">dataset description</a>, each variable has the following meaning:</p>
<p>total_bill: bill in dollars,
tip: tip in dollars,
sex: sex of the bill payer,
smoker: whether there were smokers in the party,
day: day of the week,
time: time of day,
size: size of the party.</p>
<Table
	header={['Name', 'Meaning']}
	tableRows={[
		['total_bill', 'bill in dollars'],
		['tip', 'tip in dollars'],
		['sex', 'sex of the bill payer'],
		['smoker', 'whether there were smokers in the party'],
		['day', 'day of the week'],
		['time', 'time of day'],
		['size', 'size of the party']
	]}
/>
<h3 id="scatter-plots">Scatter Plots</h3>
<p>In this example, we&rsquo;ll compare a simple scatter plot with and without <code>hue</code> to showcase how it can enhance our understanding of the data.</p>
<p>Without <code>hue</code></p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">scatterplot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&#34;total_bill&#34;</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">&#34;tip&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span></span></span></code></pre></div><p><img src="/blog/2024/hue-in-seaborn/scatterPlotWithoutHue.jpg" alt="Scatter Plot Without Hue"></p>
<p>With <code>hue</code></p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">scatterplot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&#34;total_bill&#34;</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">&#34;tip&#34;</span><span class="p">,</span> <span class="n">hue</span><span class="o">=</span><span class="s2">&#34;time&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span></span></span></code></pre></div><p><img src="/blog/2024/hue-in-seaborn/scatterPlotWithHue.jpg" alt="Scatter Plot With Hue"></p>
<p>In this example, we compare two scatter plots side by side. The left plot (without <code>hue</code>) shows a basic scatter plot of total bill amount vs. tip without distinguishing different days. The right plot (with <code>hue</code>) introduces the &lsquo;day&rsquo; column as the <code>hue</code> parameter, coloring the points based on the days of the week.</p>
<p>By using <code>hue</code>, we can observe how the relationship between total bill and tip varies across different time. This additional categorical information enhances the visualization, making it easier to differentiate patterns and trends within the data.</p>
<h3 id="bar-plots">Bar Plots</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">barplot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&#34;day&#34;</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">&#34;total_bill&#34;</span><span class="p">,</span> <span class="n">hue</span><span class="o">=</span><span class="s2">&#34;sex&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span></span></span></code></pre></div><p><img src="/blog/2024/hue-in-seaborn/barPlotWithHue.jpg" alt="Bar Plot With Hue"></p>
<h3 id="box-plots">Box Plots</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">boxplot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&#34;day&#34;</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">&#34;total_bill&#34;</span><span class="p">,</span> <span class="n">hue</span><span class="o">=</span><span class="s2">&#34;sex&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span></span></span></code></pre></div><p><img src="/blog/2024/hue-in-seaborn/boxPlotWithHue.jpg" alt="Box Plot With Hue"></p>
<h3 id="violin-plots">Violin Plots</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">violinplot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&#34;day&#34;</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">&#34;total_bill&#34;</span><span class="p">,</span> <span class="n">hue</span><span class="o">=</span><span class="s2">&#34;sex&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span></span></span></code></pre></div><p><img src="/blog/2024/hue-in-seaborn/violinPlotWithHue.jpg" alt="Violin Plot With Hue"></p>
<h3 id="count-plots">Count Plots</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">countplot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&#34;day&#34;</span><span class="p">,</span> <span class="n">hue</span><span class="o">=</span><span class="s2">&#34;sex&#34;</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span></span></span></code></pre></div><p><img src="/blog/2024/hue-in-seaborn/countPlotWithHue.jpg" alt="Count Plot With Hue"></p>
<h2 id="-when-to-use-and-when-not-to-use-hue">🤔 When to Use and When Not to Use <code>hue</code>?</h2>
<p>The use of <code>hue</code> is particularly beneficial in the following cases:</p>
<ul>
<li>
<p><strong>Visualizing Categorical Relationships:</strong>
When you are visualizing relationships between two numeric variables and have a third categorical variable that you want to explore simultaneously.</p>
</li>
<li>
<p><strong>Highlighting Patterns or Trends:</strong>
When you want to highlight patterns, trends, or differences in your data by using different colors for different categories.</p>
</li>
<li>
<p><strong>Enhancing Interpretability:</strong>
When you want to enhance the interpretability of your visualizations by adding an extra layer of information through color.</p>
</li>
</ul>
<p>While the <code>hue</code> parameter in Seaborn can be a powerful tool for enhancing visualizations by incorporating an additional categorical variable, there are situations where it might not be necessary or could potentially lead to confusion:</p>
<ul>
<li>
<p><strong>Limited or No Categorical Data :</strong>
If your dataset has very limited categorical data or if the categorical variable doesn&rsquo;t provide meaningful insights into the patterns you are exploring, using <code>hue</code> might not add much value.
When the variable specified in <code>hue</code> is not categorical, but rather a continuous variable, using <code>hue</code> may not be appropriate. In such cases, alternative approaches like color gradients or size encoding might be more suitable.</p>
</li>
<li>
<p><strong>Overcrowded Plots:</strong>
When the number of categories within the <code>hue</code> variable is large, it can lead to overcrowded plots with too many colors, making it difficult to distinguish and interpret the information.</p>
</li>
<li>
<p><strong>Color Vision Impairments:</strong>
Consider the audience of your visualizations. If there is a possibility that the audience includes individuals with color vision impairments, relying heavily on color differentiation (such as with <code>hue</code>) might limit accessibility. Ensure your visualizations are still interpretable in grayscale.</p>
</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>In conclusion, Seaborn&rsquo;s <code>hue</code> parameter is a dynamic tool that breathes vitality into visualizations, providing nuanced insights across plot types, from scatter plots to count plots. By introducing an additional layer of categorical information, <code>hue</code> unveils hidden patterns and relationships. However, its use should be judicious, especially when simplicity is crucial.</p>
]]></content:encoded></item><item><title>Math Typesetting in Astro .mdx: Using remark-math &amp; rehype-katex</title><link>https://www.byteli.com/blog/2024/math_in_astro/</link><pubDate>Wed, 17 Jan 2024 11:18:10 +0000</pubDate><author>binli@byteli.com (Bin Li)</author><guid>https://www.byteli.com/blog/2024/math_in_astro/</guid><description>&lt;p>Photo by &lt;a href="https://unsplash.com/@artturijalli?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Artturi Jalli&lt;/a> on &lt;a href="https://unsplash.com/photos/a-chalkboard-with-some-writing-on-it-gYrYa37fAKI?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash&lt;/a>&lt;/p>
&lt;p>Mathematical expressions play a crucial role in conveying complex ideas, especially in technical content. In this blog, we&amp;rsquo;ll explore how to enable math typesetting in Astro .mdx files using the powerful combination of &lt;code>remark-math&lt;/code> and &lt;code>rehype-katex&lt;/code>.&lt;/p>
&lt;p>&lt;img src="https://www.byteli.com/blog/2024/math_in_astro/astro_with_math_typesetting.jpg" alt="astro_with_math_typesetting">&lt;/p>
&lt;h2 id="step-1-package-installation">Step 1: Package installation&lt;/h2>
&lt;p>To kickstart the process, we need to install the necessary packages – &lt;code>remark-math&lt;/code> and &lt;code>rehype-katex&lt;/code>. Open your terminal and run the following command:&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-shell" data-lang="shell">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">$ npm install remark-math rehype-katex&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;ul>
&lt;li>
&lt;p>&lt;code>remark-math&lt;/code>: This package extends Markdown&amp;rsquo;s capabilities, allowing seamless support for mathematical expressions.&lt;/p></description><content:encoded><![CDATA[<p>Photo by <a href="https://unsplash.com/@artturijalli?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Artturi Jalli</a> on <a href="https://unsplash.com/photos/a-chalkboard-with-some-writing-on-it-gYrYa37fAKI?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a></p>
<p>Mathematical expressions play a crucial role in conveying complex ideas, especially in technical content. In this blog, we&rsquo;ll explore how to enable math typesetting in Astro .mdx files using the powerful combination of <code>remark-math</code> and <code>rehype-katex</code>.</p>
<p><img src="/blog/2024/math_in_astro/astro_with_math_typesetting.jpg" alt="astro_with_math_typesetting"></p>
<h2 id="step-1-package-installation">Step 1: Package installation</h2>
<p>To kickstart the process, we need to install the necessary packages – <code>remark-math</code> and <code>rehype-katex</code>. Open your terminal and run the following command:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="ln">1</span><span class="cl">$ npm install remark-math rehype-katex</span></span></code></pre></div><ul>
<li>
<p><code>remark-math</code>: This package extends Markdown&rsquo;s capabilities, allowing seamless support for mathematical expressions.</p>
</li>
<li>
<p><code>rehype-katex</code>: Specifically designed to work with KaTeX, this package handles the rendering of mathematical expressions in Markdown files with speed and efficiency.</p>
</li>
</ul>
<h2 id="step-2-modifying-the-astro-config-file">Step 2: Modifying the Astro config file</h2>
<p>Now, let&rsquo;s modify the Astro config file (<code>astro.config.ts</code> or <code>astro.config.mjs</code>) to incorporate these packages. Import and enable <code>remark-math</code> and <code>rehype-katex</code> as shown below:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ts" data-lang="ts"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="kr">import</span> <span class="p">{</span> <span class="nx">defineConfig</span> <span class="p">}</span> <span class="kr">from</span> <span class="s1">&#39;astro/config&#39;</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="kr">import</span> <span class="nx">remarkMath</span> <span class="kr">from</span> <span class="s1">&#39;remark-math&#39;</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="kr">import</span> <span class="nx">rehypeKatex</span> <span class="kr">from</span> <span class="s1">&#39;rehype-katex&#39;</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="kr">export</span> <span class="k">default</span> <span class="nx">defineConfig</span><span class="p">({</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">	<span class="c1">//
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="c1"></span>	<span class="nx">markdown</span><span class="o">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl">		<span class="nx">remarkPlugins</span><span class="o">:</span> <span class="p">[</span><span class="nx">remarkMath</span><span class="p">],</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl">		<span class="nx">rehypePlugins</span><span class="o">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl">			<span class="p">[</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl">				<span class="nx">rehypeKatex</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl">				<span class="p">{</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl">					<span class="c1">// Katex plugin options
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="c1"></span>				<span class="p">}</span>
</span></span><span class="line"><span class="ln">15</span><span class="cl">			<span class="p">]</span>
</span></span><span class="line"><span class="ln">16</span><span class="cl">		<span class="p">]</span>
</span></span><span class="line"><span class="ln">17</span><span class="cl">	<span class="p">}</span>
</span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="p">})</span></span></span></code></pre></div><h2 id="step-3-use-auto-render-extension-in-katex">Step 3: Use Auto-render Extension in Katex</h2>
<p>To render math across the entire HTML page, we&rsquo;ll add the Auto-render Extension of KaTeX in the head tag of our Astro layout file for Markdown pages. Locate or create the layout file under <code>./src/layouts/</code>. Find the latest Katex code snippet on <a href="https://katex.org/docs/autorender">Auto-render Extension</a>.</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-html" data-lang="html"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="p">&lt;</span><span class="nt">html</span> <span class="na">lang</span><span class="o">=</span><span class="s">&#34;en&#34;</span> <span class="na">class</span><span class="o">=</span><span class="s">&#34;scroll-smooth&#34;</span><span class="p">&gt;</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl">	<span class="p">&lt;</span><span class="nt">head</span><span class="p">&gt;</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">		<span class="c">&lt;!-- Katex --&gt;</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">		<span class="p">&lt;</span><span class="nt">link</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl">			<span class="na">rel</span><span class="o">=</span><span class="s">&#34;stylesheet&#34;</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">			<span class="na">href</span><span class="o">=</span><span class="s">&#34;https://cdn.jsdelivr.net/npm/katex@0.16.9/dist/katex.min.css&#34;</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl">			<span class="na">integrity</span><span class="o">=</span><span class="s">&#34;sha384-n8MVd4RsNIU0tAv4ct0nTaAbDJwPJzDEaqSD1odI+WdtXRGWt2kTvGFasHpSy3SV&#34;</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl">			<span class="na">crossorigin</span><span class="o">=</span><span class="s">&#34;anonymous&#34;</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl">		<span class="p">/&gt;</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl">		<span class="p">&lt;</span><span class="nt">script</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl">			<span class="na">defer</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl">			<span class="na">src</span><span class="o">=</span><span class="s">&#34;https://cdn.jsdelivr.net/npm/katex@0.16.9/dist/katex.min.js&#34;</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl">			<span class="na">integrity</span><span class="o">=</span><span class="s">&#34;sha384-XjKyOOlGwcjNTAIQHIpgOno0Hl1YQqzUOEleOLALmuqehneUG+vnGctmUb0ZY0l8&#34;</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl">			<span class="na">crossorigin</span><span class="o">=</span><span class="s">&#34;anonymous&#34;</span>
</span></span><span class="line"><span class="ln">15</span><span class="cl">		<span class="p">&gt;&lt;/</span><span class="nt">script</span><span class="p">&gt;</span>
</span></span><span class="line"><span class="ln">16</span><span class="cl">		<span class="p">&lt;</span><span class="nt">script</span>
</span></span><span class="line"><span class="ln">17</span><span class="cl">			<span class="na">defer</span>
</span></span><span class="line"><span class="ln">18</span><span class="cl">			<span class="na">src</span><span class="o">=</span><span class="s">&#34;https://cdn.jsdelivr.net/npm/katex@0.16.9/dist/contrib/auto-render.min.js&#34;</span>
</span></span><span class="line"><span class="ln">19</span><span class="cl">			<span class="na">integrity</span><span class="o">=</span><span class="s">&#34;sha384-+VBxd3r6XgURycqtZ117nYw44OOcIax56Z4dCRWbxyPt0Koah1uHoK0o4+/RRE05&#34;</span>
</span></span><span class="line"><span class="ln">20</span><span class="cl">			<span class="na">crossorigin</span><span class="o">=</span><span class="s">&#34;anonymous&#34;</span>
</span></span><span class="line"><span class="ln">21</span><span class="cl">			<span class="na">onload</span><span class="o">=</span><span class="s">&#34;renderMathInElement(document.body);&#34;</span>
</span></span><span class="line"><span class="ln">22</span><span class="cl">		<span class="p">&gt;&lt;/</span><span class="nt">script</span><span class="p">&gt;</span>
</span></span><span class="line"><span class="ln">23</span><span class="cl">	<span class="p">&lt;/</span><span class="nt">head</span><span class="p">&gt;</span>
</span></span><span class="line"><span class="ln">24</span><span class="cl">
</span></span><span class="line"><span class="ln">25</span><span class="cl">	<span class="p">&lt;</span><span class="nt">body</span><span class="p">&gt;</span>
</span></span><span class="line"><span class="ln">26</span><span class="cl">		<span class="p">&lt;</span><span class="nt">slot</span> <span class="p">/&gt;</span>
</span></span><span class="line"><span class="ln">27</span><span class="cl">	<span class="p">&lt;/</span><span class="nt">body</span><span class="p">&gt;</span>
</span></span><span class="line"><span class="ln">28</span><span class="cl"><span class="p">&lt;/</span><span class="nt">html</span><span class="p">&gt;</span></span></span></code></pre></div><h2 id="conclusion">Conclusion</h2>
<p>Enabling math typesetting in Astro .mdx files is a streamlined process with the powerful combination of <code>remark-math</code> and <code>rehype-katex</code>. This integration enhances your content, allowing you to seamlessly include and beautifully render mathematical expressions, providing clarity and precision to your readers. Whether you&rsquo;re a technical author or anyone dealing with mathematical content, this approach ensures a smooth workflow and an enhanced reading experience.</p>
]]></content:encoded></item><item><title>Exploring CDF vs PPF in SciPy: Understanding Probability Functions</title><link>https://www.byteli.com/blog/2024/cdf-and-ppf-in-in-scipy/</link><pubDate>Mon, 08 Jan 2024 14:34:27 +0000</pubDate><author>binli@byteli.com (Bin Li)</author><guid>https://www.byteli.com/blog/2024/cdf-and-ppf-in-in-scipy/</guid><description>&lt;p>Photo by &lt;a href="https://unsplash.com/@stumpie10?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Robert Stump&lt;/a> on &lt;a href="https://unsplash.com/photos/red-and-white-dice-lot-pQyTChJwEDI?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash&lt;/a>&lt;/p>
&lt;h2 id="introduction">Introduction&lt;/h2>
&lt;p>In probability theory, the Probability Point Function (PPF) and Cumulative Distribution Function (CDF) serve as fundamental tools in understanding and quantifying uncertainty within random variables. In this blog, we delve into the significance, applications, and practical implementation of PPF and CDF by analyzing the daily return of Apple&amp;rsquo;s stock price in 2023. Meanwhile, we will introduce &lt;a href="https://scipy.org/">SciPy&lt;/a>, an open-source Python library designed for scientific and technical computing.&lt;/p></description><content:encoded><![CDATA[<p>Photo by <a href="https://unsplash.com/@stumpie10?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Robert Stump</a> on <a href="https://unsplash.com/photos/red-and-white-dice-lot-pQyTChJwEDI?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a></p>
<h2 id="introduction">Introduction</h2>
<p>In probability theory, the Probability Point Function (PPF) and Cumulative Distribution Function (CDF) serve as fundamental tools in understanding and quantifying uncertainty within random variables. In this blog, we delve into the significance, applications, and practical implementation of PPF and CDF by analyzing the daily return of Apple&rsquo;s stock price in 2023. Meanwhile, we will introduce <a href="https://scipy.org/">SciPy</a>, an open-source Python library designed for scientific and technical computing.</p>
<h2 id="cumulative-distribution-function-cdf">Cumulative Distribution Function (CDF)</h2>
<h3 id="definition">Definition</h3>
<p>The Cumulative Distribution Function (CDF) is a function that describes the probability distribution of a random variable by specifying the probability that the variable will be less than or equal to a certain value. In simpler terms, it gives the probability of a random variable taking on a value less than or equal to a specified number.</p>
<p>Mathematically, for a random variable X, the CDF is denoted as F(x) and is expressed as:</p>
<p>$ F(x) = P(X \leq x)$, for all $x \in \R $</p>
<h2 id="probability-point-function-ppf-or-inverse-cumulative-distribution-function-cdf">Probability Point Function (PPF) or Inverse Cumulative Distribution Function (CDF)</h2>
<h3 id="definition-1">Definition</h3>
<p>The Probability Point Function (PPF), also known as the inverse cumulative distribution function, operates inversely to the CDF. It takes a probability value as input and returns the corresponding value of the random variable for which the CDF equals that probability.</p>
<p>Mathematically, if F(x) is the CDF of a random variable X, then the PPF is denoted as $F^{-1}(p)$ and is expressed as:</p>
<p>$F^{-1}(p)= x$, such that $F(x) = p$</p>
<p>The PPF is particularly useful in statistics for determining values associated with specific probabilities, such as percentiles or critical values.</p>
<h2 id="contrasting-ppf-and-cdf">Contrasting PPF and CDF</h2>
<ul>
<li>
<p><strong>Functionality</strong></p>
<p>CDF provides the probability of the variable being less than or equal to x;</p>
<p>PPF helps to find the value associated with a given probability.</p>
</li>
<li>
<p><strong>Usage</strong></p>
<p>CDF is fundamental for understanding the overall behavior of a probability distribution.</p>
<p>PPF mainly used for determining critical values or thresholds.</p>
</li>
<li>
<p><strong>Representation</strong></p>
<p>CDF: Graphically represents the cumulative probability distribution, usually starting from 0 to 1, illustrating how probabilities accumulate as values increase.</p>
<p>PPF: Graphically represents the inverse of the cumulative distribution function, plotting the values of the random variable against probabilities, showing the values associated with specific probabilities.</p>
</li>
</ul>
<h2 id="practical-example-in-scipy">Practical Example in <code>Scipy</code></h2>
<p>In this example, we&rsquo;ll demonstrate the concepts of PPF and CDF within SciPy. We&rsquo;ll use the daily returns of Apple (AAPL) in 2023 as our dataset.</p>
<h3 id="get-stock-return-data-using-yfinance">Get Stock Return Data Using <code>yfinance</code></h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="c1"># For getting historical financial data</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="kn">import</span> <span class="nn">yfinance</span> <span class="k">as</span> <span class="nn">yf</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="c1"># For scientific computing in statistics</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="kn">from</span> <span class="nn">scipy.stats</span> <span class="kn">import</span> <span class="n">norm</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="c1"># For plotting and visualization</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl">
</span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="c1"># Fetch AAPL data</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="n">stock_data</span> <span class="o">=</span> <span class="n">yf</span><span class="o">.</span><span class="n">download</span><span class="p">(</span><span class="s1">&#39;AAPL&#39;</span><span class="p">,</span> <span class="n">start</span><span class="o">=</span><span class="s1">&#39;2023-01-01&#39;</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s1">&#39;2023-12-31&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="c1"># Get a concise summary of our DataFrame</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="n">stock_data</span><span class="o">.</span><span class="n">info</span><span class="p">()</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl">
</span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="s2">&#34;&#34;&#34;
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="s2">&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="s2">DatetimeIndex: 250 entries, 2023-01-03 to 2023-12-29
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="s2">Data columns (total 6 columns):
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="s2"> #   Column     Non-Null Count  Dtype
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="s2">---  ------     --------------  -----
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="s2"> 0   Open       250 non-null    float64
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="s2"> 1   High       250 non-null    float64
</span></span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="s2"> 2   Low        250 non-null    float64
</span></span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="s2"> 3   Close      250 non-null    float64
</span></span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="s2"> 4   Adj Close  250 non-null    float64
</span></span></span><span class="line"><span class="ln">24</span><span class="cl"><span class="s2"> 5   Volume     250 non-null    int64
</span></span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="s2">dtypes: float64(5), int64(1)
</span></span></span><span class="line"><span class="ln">26</span><span class="cl"><span class="s2">memory usage: 13.7 KB
</span></span></span><span class="line"><span class="ln">27</span><span class="cl"><span class="s2">&#34;&#34;&#34;</span>
</span></span><span class="line"><span class="ln">28</span><span class="cl">
</span></span><span class="line"><span class="ln">29</span><span class="cl"><span class="c1"># Get the daiily return using `pct_change()`, remove missing values</span>
</span></span><span class="line"><span class="ln">30</span><span class="cl"><span class="n">stock_returns</span> <span class="o">=</span> <span class="n">stock_data</span><span class="p">[</span><span class="s1">&#39;Adj Close&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">pct_change</span><span class="p">()</span><span class="o">.</span><span class="n">dropna</span><span class="p">()</span></span></span></code></pre></div><h3 id="inspect-the-distribution-by-plotting-histogram-of-daily-returns">Inspect the distribution by plotting histogram of daily returns</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln">1</span><span class="cl"><span class="c1"># Inspect the distribution by plotting histogram of daily returns</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">8</span><span class="p">,</span> <span class="mi">6</span><span class="p">))</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">hist</span><span class="p">(</span><span class="n">stock_returns</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">50</span><span class="p">,</span> <span class="n">density</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">0.7</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="s1">&#39;skyblue&#39;</span><span class="p">,</span> <span class="n">edgecolor</span><span class="o">=</span><span class="s1">&#39;black&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;Histogram of Apple Inc. (AAPL)  Daily Returns in 2023&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s1">&#39;Daily Returns(%)&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s1">&#39;Frequency&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">7</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">grid</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">8</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span></span></span></code></pre></div><p><img src="/blog/2024/cdf-and-ppf-in-in-scipy/hist.png" alt="hist"></p>
<h3 id="construct-the-norm-distribution-in-scipy">Construct the norm distribution in <code>SciPy</code></h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln">1</span><span class="cl"><span class="c1"># Calculate mean and standard deviation of daily stock returns</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="n">mean_return</span> <span class="o">=</span> <span class="n">stock_returns</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="n">std_deviation</span> <span class="o">=</span> <span class="n">stock_returns</span><span class="o">.</span><span class="n">std</span><span class="p">()</span>
</span></span><span class="line"><span class="ln">4</span><span class="cl">
</span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="c1"># Create a normal distribution based on the calculated mean and standard deviation</span>
</span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="n">appl_daily_return_distribution</span> <span class="o">=</span> <span class="n">norm</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="n">mean_return</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="n">std_deviation</span><span class="p">)</span></span></span></code></pre></div><h3 id="calculating-cdf-and-ppf-in-scipy">Calculating CDF and PPF in <code>SciPy</code></h3>
<p>The <code>.cdf()</code> and <code>.ppf()</code> methods in SciPy are essential functionalities within the <code>scipy.stats</code> module that handle Cumulative Distribution Function (CDF) and Probability Point Function (PPF), respectively, for various probability distributions.</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="c1"># CDF</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="n">appl_daily_return_distribution</span><span class="o">.</span><span class="n">cdf</span><span class="p">(</span><span class="mf">0.01</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">
</span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="c1"># 0.7420136860623278</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="c1"># 74.20% of the daily returns of Apple is less or equal to 1%</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="n">appl_daily_return_distribution</span><span class="o">.</span><span class="n">cdf</span><span class="p">(</span><span class="mf">0.005</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="c1"># 0.5994001904523659</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="c1"># 59.94% of the daily returns of Apple is less or equal to 0.5%</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl">
</span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="n">appl_daily_return_distribution</span><span class="o">.</span><span class="n">cdf</span><span class="p">(</span><span class="mf">0.01</span><span class="p">)</span> <span class="o">-</span> <span class="n">appl_daily_return_distribution</span><span class="o">.</span><span class="n">cdf</span><span class="p">(</span><span class="mf">0.005</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl">
</span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="c1"># 0.14261349560996184</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="c1"># 14.26% of the daily returns of Apple is greater than 0.5% and less or equal to 1%</span></span></span></code></pre></div>




<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="c1"># PPF</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="n">appl_daily_return_distribution</span><span class="o">.</span><span class="n">ppf</span><span class="p">(</span><span class="mf">0.9</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">
</span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="c1"># 0.017944086843352632</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="c1"># The value of which 90% of all the daily returns are less or equal to is 1.79%</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="n">appl_daily_return_distribution</span><span class="o">.</span><span class="n">ppf</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="mf">0.9</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="c1"># -0.014274232117790355</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="c1"># The value of which 90% of all the daily returns are less or equal to is -1.42%</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="c1"># In other words, the value of which 90% of all the daily returns are greater to is -1.42%.</span></span></span></code></pre></div><h3 id="visualize-the-distribution">Visualize the Distribution</h3>
<h4 id="cdf">CDF</h4>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="c1"># Calculate the CDF for a range of values</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="n">x_values</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">stock_returns</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="n">y_cdf</span> <span class="o">=</span> <span class="n">appl_daily_return_distribution</span><span class="o">.</span><span class="n">cdf</span><span class="p">(</span><span class="n">x_values</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="c1"># Plotting the Cumulative Distribution Function (CDF)</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">8</span><span class="p">,</span> <span class="mi">6</span><span class="p">))</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x_values</span><span class="p">,</span> <span class="n">y_cdf</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;CDF&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">&#34;Cumulative Distribution Function (CDF) of Apple&#39;s Stock Daily Returns in 2023&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s1">&#39;Daily Returns(%)&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s1">&#39;Cumulative Probability&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">()</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">grid</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span></span></span></code></pre></div><p><img src="/blog/2024/cdf-and-ppf-in-in-scipy/cdf.png" alt="cdf"></p>
<h3 id="ppf">PPF</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="c1"># Calculate the PPF for a range of probabilities</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="n">probabilities</span> <span class="o">=</span> <span class="p">[</span><span class="mf">0.05</span><span class="p">,</span> <span class="mf">0.25</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">,</span> <span class="mf">0.75</span><span class="p">,</span> <span class="mf">0.95</span><span class="p">]</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="n">x_ppf</span> <span class="o">=</span> <span class="n">appl_daily_return_distribution</span><span class="o">.</span><span class="n">ppf</span><span class="p">(</span><span class="n">probabilities</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="c1"># Plotting the Probability Point Function (PPF)</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">8</span><span class="p">,</span> <span class="mi">6</span><span class="p">))</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">probabilities</span><span class="p">,</span> <span class="n">x_ppf</span><span class="p">,</span> <span class="n">marker</span><span class="o">=</span><span class="s1">&#39;o&#39;</span><span class="p">,</span> <span class="n">linestyle</span><span class="o">=</span><span class="s1">&#39;None&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;PPF&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">&#34;Probability Point Function (PPF) of of Apple&#39;s Stock Daily Returns in 2023&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s1">&#39;Probability&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s1">&#39;Stock Daily Returns(%)&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">()</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">grid</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span></span></span></code></pre></div><p><img src="/blog/2024/cdf-and-ppf-in-in-scipy/ppf.png" alt="ppf"></p>
<h2 id="conclusion">Conclusion</h2>
<p>In summary, while the CDF gives the probability that a random variable is less than or equal to a particular value, the PPF helps in finding the value of the random variable for a given probability. They are complementary functions often used together in statistical analysis and probability calculations.</p>
]]></content:encoded></item><item><title>Options Strategies: Use Protective Put to Hedge Market Downside Risk</title><link>https://www.byteli.com/blog/2023/how-to-use-options-to-hedge-market-downside-risk/</link><pubDate>Sat, 01 Jul 2023 00:00:00 +0000</pubDate><author>binli@byteli.com (Bin Li)</author><guid>https://www.byteli.com/blog/2023/how-to-use-options-to-hedge-market-downside-risk/</guid><description>&lt;p>Photo by &lt;a href="https://unsplash.com/@markusspiske?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Markus Spiske&lt;/a> on &lt;a href="https://unsplash.com/photos/black-android-smartphone-displaying-white-screen-nOv3mUv3mvA?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash&lt;/a>&lt;/p>
&lt;p>In the unpredictable world of financial markets, investors often seek strategies to protect their portfolios from potential downturns. One approach to managing downside risk is through the use of options. By employing various options strategies, investors can hedge against potential losses in their portfolios if the market experiences a decline.&lt;/p>
&lt;p>Among all the options strategies, protective put is an easy-to-construct yet effective way to hedge market downside risk while participating in the market&amp;rsquo;s potential upside. This article will explore how to construct a protective put, providing an overview of this strategy and its potential strengths and weaknesses.&lt;/p></description><content:encoded><![CDATA[<p>Photo by <a href="https://unsplash.com/@markusspiske?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Markus Spiske</a> on <a href="https://unsplash.com/photos/black-android-smartphone-displaying-white-screen-nOv3mUv3mvA?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a></p>
<p>In the unpredictable world of financial markets, investors often seek strategies to protect their portfolios from potential downturns. One approach to managing downside risk is through the use of options. By employing various options strategies, investors can hedge against potential losses in their portfolios if the market experiences a decline.</p>
<p>Among all the options strategies, protective put is an easy-to-construct yet effective way to hedge market downside risk while participating in the market&rsquo;s potential upside. This article will explore how to construct a protective put, providing an overview of this strategy and its potential strengths and weaknesses.</p>
<p><img src="/blog/2023/how-to-use-options-to-hedge-market-downside-risk/compare_stock_payoff.png" alt="Downside Risk Reduction Under Protective Put Strategy"></p>
<h2 id="introduction-to-protective-put">Introduction to Protective Put</h2>
<p>A protective put is constructed by:</p>
<ul>
<li>Holding a long position in the underlying security and</li>
<li>Buying a put option;</li>
</ul>
<p>The basic idea is when the stock price falls under the strike price, the put option&rsquo;s increasing value will hedge our loss; cause you have the right to sell the stock at a higher price(strike price).</p>
<p>As the stock price drops, so do our stock returns. However, the value of our put option increases correspondingly, hedging the downside risk and protecting us with a floor loss.</p>
<p>Hence, as shown in the diagram below, your ultimate payoff(blue line) is the combination of the payoff on the stock(green line) and the put option(orange dotted line).</p>
<p><img src="/blog/2023/how-to-use-options-to-hedge-market-downside-risk/protective_put_explained.png" alt="Explanation and Elements of a Protective Put"></p>
<h2 id="explain-with-example">Explain with Example</h2>
<p>Let&rsquo;s illustrate this strategy with an example. To simplify the calculation, we do not take transaction costs into account.</p>
<p>Suppose we buy one share of Apple&rsquo;s stock at <span>$</span>190; we want to mitigate the downside risk and implement the protective put strategy by buying a put option on Apple&rsquo;s stock for <span>$</span>2 with a strike price of <span>$</span>190.</p>
<table>
  <thead>
      <tr>
          <th>Symbol</th>
          <th>AAPL</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Buy Price of Stock</td>
          <td>$190.00</td>
      </tr>
      <tr>
          <td>Cost of Put Option (Put Premium)</td>
          <td>$2.00</td>
      </tr>
      <tr>
          <td>Strike Price</td>
          <td>$190.00</td>
      </tr>
  </tbody>
</table>
<ul>
<li>
<p>When the stock price falls under the strike price</p>
<p>Let&rsquo;s say the stock price falls to $180. In this situation:</p>
<ul>
<li>our payoff from the stock is $\$180 - \$190 = -\$10$</li>
<li>our payoff from the put option is $Max(\$190 - \$180, 0) - \$2 = \$8$</li>
<li>the total payoff is $-\$10 +  \$8 = -\$2$</li>
</ul>
<p>By doing so, we successfully limit our potential loss to -<span>$</span>2, which is the cost of the put option, aka put premium. In fact, our loss will be limited to <span>$</span>2 regardless of how much the underlying stock price falls.</p>
</li>
<li>
<p>When the stock price rises above the strike price</p>
<p>Suppose the stock price rises to $191. In this situation:</p>
<ul>
<li>the payoff from the stock is $\$191 - \$190 = \$1$</li>
<li>the payoff from the put option is $Max(\$190 - \$191, 0) - \$2 = -\$2$</li>
<li>the total payoff is $\$1 -  \$2 = -\$1$</li>
</ul>
<p>Notice that even if the current stock price is higher than our previous buy price, we&rsquo;re still losing money due to the put option&rsquo;s cost. In fact, we can only gain once the current stock price is greater than the sum of the buy price and the put premium. In other words, our break-even point in this situation is 190 + 2 = $192.</p>
<p>Assuming the stock price continues to rise to $200. In this scenario, our return is calculated as follows:</p>
<ul>
<li>the payoff from the stock is $\$200 - \$190 = \$10$</li>
<li>the payoff from the put option is $Max(\$190 - \$200, 0) - \$2 = -\$2$</li>
<li>the total payoff is $\$10 -  \$2 = \$8$</li>
</ul>
<p>We can still profit from the increase in stock price.</p>
</li>
</ul>
<h2 id="strengthens-and-weakness">Strengthens and Weakness</h2>
<p>Strengthens:</p>
<ul>
<li>
<p>Downside Protection</p>
<p>The Protective Put strategy allows investors to limit potential losses in the event of a market decline or a drop in the underlying asset&rsquo;s value while still benefiting from market upsides.</p>
</li>
<li>
<p>Simplicity</p>
<p>The Protective Put strategy is relatively straightforward to understand and implement, providing a clear and easy-to-follow approach to risk management.</p>
</li>
<li>
<p>Participation in Upside Potential</p>
<p>The investor can benefit from the appreciation in the asset&rsquo;s value while having the downside risk hedged by the put options.</p>
</li>
<li>
<p>Flexibility</p>
<p>The Protective Put strategy can be applied to various underlying assets, including stocks, ETFs, or indexes.</p>
</li>
</ul>
<p>Weakness:</p>
<ul>
<li>
<p>Limited Upside</p>
<p>The Protective Put strategy&rsquo;s primary objective is to hedge against downside risk. If the market rises, the value of the put options may decrease or expire worthless, which can offset gains in the underlying asset.</p>
</li>
<li>
<p>Cost</p>
<p>The put premium can eat into your overall returns, especially when the market is stable or only experiences a minor decline.</p>
</li>
<li>
<p>Rolling Costs</p>
<p>To maintain continuous protection, investors must roll their options positions by closing out expiring options and opening new ones. This rolling process incurs additional transaction costs, which can impact overall returns.</p>
</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>In summary, a protective put provides downside protection by setting a floor price at which you can sell your shares; the downside risk is limited at the cost of the put premium, i.e. the money we have to pay for the put option. While the Protective Put strategy offers such advantages, it&rsquo;s essential to notice that it has some weaknesses that can negatively affect the total payoff. Hence, we must assess its suitability based on investment goals, risk tolerance, and market outlook.</p>
]]></content:encoded></item><item><title>My Tips for FRM Part 2 Exam Preparation -- Passed with 111231</title><link>https://www.byteli.com/blog/2023/tips-and-tools-for-frm-part-2-exam/</link><pubDate>Sun, 25 Jun 2023 00:00:00 +0000</pubDate><author>binli@byteli.com (Bin Li)</author><guid>https://www.byteli.com/blog/2023/tips-and-tools-for-frm-part-2-exam/</guid><description>&lt;p>After serval months&amp;rsquo; preparation, I passed the part 2 exam with 111231, and I would like to share my experience preparing for the exam, hoping to give some references to others.&lt;/p>
&lt;h2 id="qualitative-questions-weigh-more-than-quantitative-ones">Qualitative questions weigh more than quantitative ones.&lt;/h2>
&lt;p>From the feedback I got from the BT forum, the P2 exam has always focused more on qualitative questions in recent years.&lt;/p>
&lt;p>So you better &lt;strong>understand the theoretical concepts&lt;/strong> in the first place rather than memorize formulas repeatedly and painfully.&lt;/p></description><content:encoded><![CDATA[<p>After serval months&rsquo; preparation, I passed the part 2 exam with 111231, and I would like to share my experience preparing for the exam, hoping to give some references to others.</p>
<h2 id="qualitative-questions-weigh-more-than-quantitative-ones">Qualitative questions weigh more than quantitative ones.</h2>
<p>From the feedback I got from the BT forum, the P2 exam has always focused more on qualitative questions in recent years.</p>
<p>So you better <strong>understand the theoretical concepts</strong> in the first place rather than memorize formulas repeatedly and painfully.</p>
<p>How to distinguish important theoretical concepts? Check the LOs.</p>
<h2 id="do-not-expect-to-memorize-and-understand-everything-at-once">Do NOT expect to memorize and understand everything at once.</h2>
<p>Leave time for the second round of study.</p>
<p>Don&rsquo;t push yourself to remember every trivial detail in the first study round, cause:</p>
<ul>
<li>You can&rsquo;t distinguish the knowledge that matters, so your memory load is unnecessarily heavy. Finish your first study round, and you&rsquo;ll get the big picture.</li>
<li>Most importantly, it will frustrate you and slow down your learning pace; at this point, some candidates may feel unmotivated and want to quit. (But not YOU.)</li>
</ul>
<p>After you finish your first study of the material, you may find yourself can barely remember what&rsquo;s in the first 2 or 3 books. That&rsquo;s how memory works, the good news is you can easily solidify those memory links in your second study round, and that&rsquo;s where &lsquo;Ah, everything is connected.&rsquo; happens most.
Personally, I leave 6 weeks for another round of review this time. I work full-time during the daytime, so I tend to leave more time for myself. Surely, this timeframe varies from person to person.</p>
<h2 id="qb-and-mocks">QB and MOCKs</h2>
<ul>
<li>
<p>QB - Collect &amp; review your mistakes regularly</p>
<p>I completed all BT&rsquo;s practice questions. But I don&rsquo;t recommend this approach, cause BT&rsquo;s practice questions are far more difficult than actual GARP&rsquo;s, especially the quantitative ones. If you just want to pass the exam, you do not need to spend soooo much time and energy.</p>
<p>However, regular review of your mistakes really helps, you may surprisingly find yourself making the same mistakes repeatedly, and you&rsquo;ll learn from them every single time.</p>
</li>
</ul>
<h2 id="mocks---do-the-garp-practice-exams">MOCKs - Do the GARP Practice Exams.</h2>
<p>I only did two GARP Practice Exams during the whole study. I think it matches the actual exam&rsquo;s difficulty and solution logic. I recommend every candidate take advantage of it, you can get it directly from GARP&rsquo;s official website, totally free.</p>
<p>When finish the mock, carefully review EVERY problem to:</p>
<ul>
<li>diminish your known unknowns and, most importantly, your unknown unknowns;</li>
<li>realize how different LOs are organized and tested as a whole;</li>
<li>follow up the solution pattern, especially the qualitative questions, cause you often can&rsquo;t solve them directly.</li>
</ul>
<h2 id="the-previous-day-before-the-exam">The previous day before the exam</h2>
<ul>
<li>Check the calculator and all your document.</li>
<li>Review all your mistakes.</li>
<li>Have a good sleep.</li>
</ul>
<h2 id="on-the-exam-day">On the exam day.</h2>
<ul>
<li>
<p>Eat breakfast (my exam was in the morning)</p>
<p>Your heart needs that energy to pump blood to your brain.</p>
</li>
<li>
<p>Bring a bottle of water.</p>
<p>Staying hydrated makes brains think better and make smarter choices.</p>
</li>
<li>
<p>Stay calm</p>
<p>Whatever questions and situations you encounter, just don&rsquo;t panic.</p>
<p>(BTW, my computer crashed TWICE on exam day.)</p>
</li>
</ul>
<h2 id="tools-i-use">Tools I use</h2>
<ul>
<li>
<p>Excel</p>
<p>Forget about fancy to-do apps, over-priced task management software, Notion, etc., excel is enough to quantify your OKRs and track your progress.</p>
</li>
<li>
<p>Anki</p>
<blockquote>
<p>Powerful, intelligent flash cards. Remembering things just became much easier. &ndash; Anki</p></blockquote>
<p><a href="https://apps.ankiweb.net/">Anki</a> is a free software(except for iOS) to make flash cards.</p>
<p>I use it to make flashcards of P2 LOs, so I can use the fragmented time, such as commuting, to memorize knowledge points in a way that conforms to the forgetting curve.</p>
</li>
<li>
<p>Pomodoro Technique</p>
<p>The <a href="https://en.wikipedia.org/wiki/Pomodoro_Technique">Pomodoro Technique</a> is a good way to deal with procrastination, greatly reducing the resistance to starting work. At the same time, it also well combines highly focused time and guilty-free breaks, which saves me a lot from a heavy study load.</p>
</li>
<li>
<p>Study Together</p>
<p><a href="https://discord.com/invite/study">Study Together</a> is a discord group, where members can join a muted study call, only opening their video cameras and study live. It&rsquo;s kinda like a virtual library, creating the study vibes whenever you need to figure the Greeks out.</p>
</li>
</ul>
]]></content:encoded></item></channel></rss>