Intro_ML

<style>

.remark-slide-number {
  position: inherit;
}

.remark-slide-number .progress-bar-container {
  position: absolute;
  bottom: 0;
  height: 4px;
  display: block;
  left: 0;
  right: 0;
}

.remark-slide-number .progress-bar {
  height: 100%;
  background-color: rgba(2, 70, 79, 0.874);
}

.remark-slide-number .numeric {
  position: absolute;
  bottom: 4%;
  height: 4px;
  display: block;
  right: 2.5%;
  font-weight: bold;
}

.remark-slide-number .numeric-out{
  color: rgba(2, 70, 79, 0.874);
}

</style>

---
count: false

<img src="img/UniversiteParisCite_logo.jpg" align="left" height="100">
<img src="img/cnrs.png" align="middle" height="90"> .white[aaaaaaa]
<img src="img/X.png" align="right" height="110" width="90"> .white[aaaaaaa]

## Introduction to Machine Learning

#### Institude de Technologie du Cambodge

### Sothea .textsc[Has], Ph.D

---
## What's Machine Learning (ML)?<hbr>
.pull-left[<hbr>
- [Arthur Samuel (1959)](https://en.wikipedia.org/wiki/Arthur_Samuel_(computer_scientist)

]

.pull-right[
<br><br><br><br>
"The field of study that gives computers
the ability to learn without being
explicitly programmed."
]

---
count:false
## What's Machine Learning (ML)?<hbr>
.pull-left[<hbr>
- [Arthur Samuel (1959)](https://en.wikipedia.org/wiki/Arthur_Samuel_(computer_scientist)

- [Tom M. Mitchell (1997)](https://en.wikipedia.org/wiki/Tom_M._Mitchell)

.pull-right[
<br><br><br><br>
"The field of study that gives computers
the ability to learn without being
explicitly programmed."

<br><br><br><br><br><hbr>
"A computer program is said to learn from experience .stress[E] with respect to some class of tasks .stress[T] and
performance measure .stress[P], if its
performance at tasks in .stress[T], as
measured by .stress[P], improves
with experience .stress[E]."
]

---
## Why do we care?<hbr> .subtitle[&nbsp; [Applications of Machine Learning](https://www.javatpoint.com/applications-of-machine-learning)]<hbr>
.center[<img src="./img/appli_ml.png" width="430px" height ="380px"/>]

Because the world is now driven by .stress[Data], and .stress[Machine Learning] is a powerful tool.

---
.pull-left[
### Traditional programming

- .stress[Programming]'s rules are observed and designed by human.
- Example: <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#74AAEC;overflow:visible;position:relative;"><path d="M64 112c-8.8 0-16 7.2-16 16v22.1L220.5 291.7c20.7 17 50.4 17 71.1 0L464 150.1V128c0-8.8-7.2-16-16-16H64zM48 212.2V384c0 8.8 7.2 16 16 16H448c8.8 0 16-7.2 16-16V212.2L322 328.8c-38.4 31.5-93.7 31.5-132 0L48 212.2zM0 128C0 92.7 28.7 64 64 64H448c35.3 0 64 28.7 64 64V384c0 35.3-28.7 64-64 64H64c-35.3 0-64-28.7-64-64V128z"/></svg> with '&' or '!' more than `$20$` times, should be a spam.
]

- .stress[Algorithms] are designed to capture the rules from the training data.
- Example: <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#74AAEC;overflow:visible;position:relative;"><path d="M64 112c-8.8 0-16 7.2-16 16v22.1L220.5 291.7c20.7 17 50.4 17 71.1 0L464 150.1V128c0-8.8-7.2-16-16-16H64zM48 212.2V384c0 8.8 7.2 16 16 16H448c8.8 0 16-7.2 16-16V212.2L322 328.8c-38.4 31.5-93.7 31.5-132 0L48 212.2zM0 128C0 92.7 28.7 64 64 64H448c35.3 0 64 28.7 64 64V384c0 35.3-28.7 64-64 64H64c-35.3 0-64-28.7-64-64V128z"/></svg> .stress[similar] to the existing spams, should also be a spam.
]

---
template: inter-slide
class: left, middle
count: false

## <svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:rgb(16, 111, 171);;overflow:visible;position:relative;"><path d="M565.6 36.2C572.1 40.7 576 48.1 576 56V392c0 10-6.2 18.9-15.5 22.4l-168 64c-5.2 2-10.9 2.1-16.1 .3L192.5 417.5l-160 61c-7.4 2.8-15.7 1.8-22.2-2.7S0 463.9 0 456V120c0-10 6.1-18.9 15.5-22.4l168-64c5.2-2 10.9-2.1 16.1-.3L383.5 94.5l160-61c7.4-2.8 15.7-1.8 22.2 2.7zM48 136.5V421.2l120-45.7V90.8L48 136.5zM360 422.7V137.3l-144-48V374.7l144 48zm48-1.5l120-45.7V90.8L408 136.5V421.2z"/></svg> .bold-blue[Outline]

<br>

<br>

<br>

<br>

---
template: inter-slide
class: left, middle
count: false

<br>

<br>

<br>

<br>

---

## I. Some Elements of Machine Learning<h0br> 
### &nbsp; 1. Data (<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#6EB741;overflow:visible;position:relative;"><path d="M32 64C32 28.7 60.7 0 96 0H256c35.3 0 64 28.7 64 64V256h8c48.6 0 88 39.4 88 88v32c0 13.3 10.7 24 24 24s24-10.7 24-24V222c-27.6-7.1-48-32.2-48-62V96L384 64c-8.8-8.8-8.8-23.2 0-32s23.2-8.8 32 0l77.3 77.3c12 12 18.7 28.3 18.7 45.3V168v24 32V376c0 39.8-32.2 72-72 72s-72-32.2-72-72V344c0-22.1-17.9-40-40-40h-8V448c17.7 0 32 14.3 32 32s-14.3 32-32 32H32c-17.7 0-32-14.3-32-32s14.3-32 32-32V64zM96 80v96c0 8.8 7.2 16 16 16H240c8.8 0 16-7.2 16-16V80c0-8.8-7.2-16-16-16H112c-8.8 0-16 7.2-16 16z"/></svg>)<h0br>

- Data `$\approx$` information, experiences, ...

- Structured data: predefined type (string, double, ...) and dimension (rows, columns, channels)...
  
  - Example: phone numbers <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#0D2746;overflow:visible;position:relative;"><path d="M164.9 24.6c-7.7-18.6-28-28.5-47.4-23.2l-88 24C12.1 30.2 0 46 0 64C0 311.4 200.6 512 448 512c18 0 33.8-12.1 38.6-29.5l24-88c5.3-19.4-4.6-39.7-23.2-47.4l-96-40c-16.3-6.8-35.2-2.1-46.3 11.6L304.7 368C234.3 334.7 177.3 277.7 144 207.3L193.3 167c13.7-11.2 18.4-30 11.6-46.3l-40-96z"/></svg>, zip codes <svg aria-hidden="true" role="img" viewBox="0 0 384 512" style="height:1em;width:0.75em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#167B64;overflow:visible;position:relative;"><path d="M384 192c0 87.4-117 243-168.3 307.2c-12.3 15.3-35.1 15.3-47.4 0C117 435 0 279.4 0 192C0 86 86 0 192 0S384 86 384 192z"/></svg>, identity number <svg aria-hidden="true" role="img" viewBox="0 0 384 512" style="height:1em;width:0.75em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#F54C27;overflow:visible;position:relative;"><path d="M256 48V64c0 17.7-14.3 32-32 32H160c-17.7 0-32-14.3-32-32V48H64c-8.8 0-16 7.2-16 16V448c0 8.8 7.2 16 16 16H320c8.8 0 16-7.2 16-16V64c0-8.8-7.2-16-16-16H256zM0 64C0 28.7 28.7 0 64 0H320c35.3 0 64 28.7 64 64V448c0 35.3-28.7 64-64 64H64c-35.3 0-64-28.7-64-64V64zM160 320h64c44.2 0 80 35.8 80 80c0 8.8-7.2 16-16 16H96c-8.8 0-16-7.2-16-16c0-44.2 35.8-80 80-80zm-32-96a64 64 0 1 1 128 0 64 64 0 1 1 -128 0z"/></svg>, gender <svg aria-hidden="true" role="img" viewBox="0 0 320 512" style="height:1em;width:0.62em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#1C2B45;overflow:visible;position:relative;"><path d="M96 64a64 64 0 1 1 128 0A64 64 0 1 1 96 64zm48 320v96c0 17.7-14.3 32-32 32s-32-14.3-32-32V287.8L59.1 321c-9.4 15-29.2 19.4-44.1 10S-4.5 301.9 4.9 287l39.9-63.3C69.7 184 113.2 160 160 160s90.3 24 115.2 63.6L315.1 287c9.4 15 4.9 34.7-10 44.1s-34.7 4.9-44.1-10L240 287.8V480c0 17.7-14.3 32-32 32s-32-14.3-32-32V384H144z"/></svg>...<hbr>

```r
spam <- read_delim(file=str_c(fpath,"spam.txt", sep = ''), show_col_types = FALSE)
spam[c(2,4061),c("receive","address","num000","type")] %>%
  knitr::kable(format = "markdown")
```

]

| receive| address| num000|type    |
|-------:|-------:|------:|:-------|
|    0.21|    0.28|   0.43|spam    |
|    0.00|    0.00|   0.00|nonspam |

]

- Unstructured data: everything else (no predefined type nor dimension)...
  
  - Example: emails <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#74AAEC;overflow:visible;position:relative;"><path d="M64 112c-8.8 0-16 7.2-16 16v22.1L220.5 291.7c20.7 17 50.4 17 71.1 0L464 150.1V128c0-8.8-7.2-16-16-16H64zM48 212.2V384c0 8.8 7.2 16 16 16H448c8.8 0 16-7.2 16-16V212.2L322 328.8c-38.4 31.5-93.7 31.5-132 0L48 212.2zM0 128C0 92.7 28.7 64 64 64H448c35.3 0 64 28.7 64 64V384c0 35.3-28.7 64-64 64H64c-35.3 0-64-28.7-64-64V128z"/></svg>, social media posts <svg aria-hidden="true" role="img" viewBox="0 0 320 512" style="height:1em;width:0.62em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#2187CB;overflow:visible;position:relative;"><path d="M279.14 288l14.22-92.66h-88.91v-60.13c0-25.35 12.42-50.06 52.24-50.06h40.42V6.26S260.43 0 225.36 0c-73.22 0-121.08 44.38-121.08 124.72v70.62H22.89V288h81.39v224h100.17V288z"/></svg>, video <svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#F22121;overflow:visible;position:relative;"><path d="M549.655 124.083c-6.281-23.65-24.787-42.276-48.284-48.597C458.781 64 288 64 288 64S117.22 64 74.629 75.486c-23.497 6.322-42.003 24.947-48.284 48.597-11.412 42.867-11.412 132.305-11.412 132.305s0 89.438 11.412 132.305c6.281 23.65 24.787 41.5 48.284 47.821C117.22 448 288 448 288 448s170.78 0 213.371-11.486c23.497-6.321 42.003-24.171 48.284-47.821 11.412-42.867 11.412-132.305 11.412-132.305s0-89.438-11.412-132.305zm-317.51 213.508V175.185l142.739 81.205-142.739 81.201z"/></svg>, sensor <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#0F4E9A;overflow:visible;position:relative;"><path d="M149.1 64.8L138.7 96H64C28.7 96 0 124.7 0 160V416c0 35.3 28.7 64 64 64H448c35.3 0 64-28.7 64-64V160c0-35.3-28.7-64-64-64H373.3L362.9 64.8C356.4 45.2 338.1 32 317.4 32H194.6c-20.7 0-39 13.2-45.5 32.8zM256 192a96 96 0 1 1 0 192 96 96 0 1 1 0-192z"/></svg> data...

- Data pre-processing : data types, removing outliers, .stress[imputing] missing values (`NA`), scaling (.stress[standardization] or .stress[normalization]),...

- .stress[GIGO] : "garbage in, garbage out!"

---

```r
timeSeries %>%
  ggplot(aes(x = time)) +
  geom_line(aes(y = win), color = '#4109F0') +
  geom_line(aes(y = win1), color = '#E71F1F') +
  geom_line(aes(y = win2), color = '#F7D606') +
  geom_line(aes(y = win3), color = '#1EC826') +
  geom_line(aes(y = win4), color = '#06B6D9') +
  geom_line(aes(y = win5), color = '#B40EE5') +
  theme(text = element_text(size = 18))
```

<hbr>

---

## I. Some Elements of Machine Learning<h0br> 
### &nbsp; 2. Task (<svg aria-hidden="true" role="img" viewBox="0 0 384 512" style="height:1em;width:0.75em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#6EB741;overflow:visible;position:relative;"><path d="M384 192c0 87.4-117 243-168.3 307.2c-12.3 15.3-35.1 15.3-47.4 0C117 435 0 279.4 0 192C0 86 86 0 192 0S384 86 384 192z"/></svg>?)<h0br>

.center[<img src="./img/ML_branches.png" width="630px" height ="450px"/><hbr>
.caption[Source: [https://askdatascience.com/13/what-are-the-main-branches-of-machine-learning](https://askdatascience.com/13/what-are-the-main-branches-of-machine-learning)]]

---
exclude:true
## I. Some Elements of Machine Learning<h0br> 
### &nbsp; 2. Task (With fuel, where do you want go?)<h0br>

.pull-left[
- Supervised learning:
  
  - Identify email (spam or not)
  
  - Weather forecast
  
  - Health care system
  
  - Increase of income from ads...
  
- Unsupervised learning:
  
  - City planning
  
  - Clustering customers or users
  
  - Organizing products
  
  - Gene representation...
]

- Reinforcement learning:
  
  - Self-driving car,
  
  - Gaming ([Alpha GO](https://www.deepmind.com/research/highlighted-research/alphago))
  
  - Robotic
  
  - Natural Language Processing (NLP)...
]

---
## I. Some Elements of Machine Learning<h0br> 
### &nbsp; 3. Model (<svg aria-hidden="true" role="img" viewBox="0 0 640 512" style="height:1em;width:1.25em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#6EB741;overflow:visible;position:relative;"><path d="M280 32c-13.3 0-24 10.7-24 24s10.7 24 24 24h57.7l16.4 30.3L256 192l-45.3-45.3c-12-12-28.3-18.7-45.3-18.7H64c-17.7 0-32 14.3-32 32v32h96c88.4 0 160 71.6 160 160c0 11-1.1 21.7-3.2 32h70.4c-2.1-10.3-3.2-21-3.2-32c0-52.2 25-98.6 63.7-127.8l15.4 28.6C402.4 276.3 384 312 384 352c0 70.7 57.3 128 128 128s128-57.3 128-128s-57.3-128-128-128c-13.5 0-26.5 2.1-38.7 6L418.2 128H480c17.7 0 32-14.3 32-32V64c0-17.7-14.3-32-32-32H459.6c-7.5 0-14.7 2.6-20.5 7.4L391.7 78.9l-14-26c-7-12.9-20.5-21-35.2-21H280zM462.7 311.2l28.2 52.2c6.3 11.7 20.9 16 32.5 9.7s16-20.9 9.7-32.5l-28.2-52.2c2.3-.3 4.7-.4 7.1-.4c35.3 0 64 28.7 64 64s-28.7 64-64 64s-64-28.7-64-64c0-15.5 5.5-29.7 14.7-40.8zM187.3 376c-9.5 23.5-32.5 40-59.3 40c-35.3 0-64-28.7-64-64s28.7-64 64-64c26.9 0 49.9 16.5 59.3 40h66.4C242.5 268.8 190.5 224 128 224C57.3 224 0 281.3 0 352s57.3 128 128 128c62.5 0 114.5-44.8 125.8-104H187.3zM128 384a32 32 0 1 0 0-64 32 32 0 1 0 0 64z"/></svg>(<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:rgb(233, 165, 9);;overflow:visible;position:relative;"><path d="M336 352c97.2 0 176-78.8 176-176S433.2 0 336 0S160 78.8 160 176c0 18.7 2.9 36.8 8.3 53.7L7 391c-4.5 4.5-7 10.6-7 17v80c0 13.3 10.7 24 24 24h80c13.3 0 24-10.7 24-24V448h40c13.3 0 24-10.7 24-24V384h40c6.4 0 12.5-2.5 17-7l33.3-33.3c16.9 5.4 35 8.3 53.7 8.3zM376 96a40 40 0 1 1 0 80 40 40 0 1 1 0-80z"/></svg>), <svg aria-hidden="true" role="img" viewBox="0 0 640 512" style="height:1em;width:1.25em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#6EB741;overflow:visible;position:relative;"><path d="M171.3 96H224v96H111.3l30.4-75.9C146.5 104 158.2 96 171.3 96zM272 192V96h81.2c9.7 0 18.9 4.4 25 12l67.2 84H272zm256.2 1L428.2 68c-18.2-22.8-45.8-36-75-36H171.3c-39.3 0-74.6 23.9-89.1 60.3L40.6 196.4C16.8 205.8 0 228.9 0 256V368c0 17.7 14.3 32 32 32H65.3c7.6 45.4 47.1 80 94.7 80s87.1-34.6 94.7-80H385.3c7.6 45.4 47.1 80 94.7 80s87.1-34.6 94.7-80H608c17.7 0 32-14.3 32-32V320c0-65.2-48.8-119-111.8-127zM434.7 368a48 48 0 1 1 90.5 32 48 48 0 1 1 -90.5-32zM160 336a48 48 0 1 1 0 96 48 48 0 1 1 0-96z"/></svg>(<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:rgb(233, 165, 9);;overflow:visible;position:relative;"><path d="M336 352c97.2 0 176-78.8 176-176S433.2 0 336 0S160 78.8 160 176c0 18.7 2.9 36.8 8.3 53.7L7 391c-4.5 4.5-7 10.6-7 17v80c0 13.3 10.7 24 24 24h80c13.3 0 24-10.7 24-24V448h40c13.3 0 24-10.7 24-24V384h40c6.4 0 12.5-2.5 17-7l33.3-33.3c16.9 5.4 35 8.3 53.7 8.3zM376 96a40 40 0 1 1 0 80 40 40 0 1 1 0-80z"/></svg>), <svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#6EB741;overflow:visible;position:relative;"><path d="M482.3 192c34.2 0 93.7 29 93.7 64c0 36-59.5 64-93.7 64l-116.6 0L265.2 495.9c-5.7 10-16.3 16.1-27.8 16.1l-56.2 0c-10.6 0-18.3-10.2-15.4-20.4l49-171.6L112 320 68.8 377.6c-3 4-7.8 6.4-12.8 6.4l-42 0c-7.8 0-14-6.3-14-14c0-1.3 .2-2.6 .5-3.9L32 256 .5 145.9c-.4-1.3-.5-2.6-.5-3.9c0-7.8 6.3-14 14-14l42 0c5 0 9.8 2.4 12.8 6.4L112 192l102.9 0-49-171.6C162.9 10.2 170.6 0 181.2 0l56.2 0c11.5 0 22.1 6.2 27.8 16.1L365.7 192l116.6 0z"/></svg>(<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:rgb(233, 165, 9);;overflow:visible;position:relative;"><path d="M336 352c97.2 0 176-78.8 176-176S433.2 0 336 0S160 78.8 160 176c0 18.7 2.9 36.8 8.3 53.7L7 391c-4.5 4.5-7 10.6-7 17v80c0 13.3 10.7 24 24 24h80c13.3 0 24-10.7 24-24V448h40c13.3 0 24-10.7 24-24V384h40c6.4 0 12.5-2.5 17-7l33.3-33.3c16.9 5.4 35 8.3 53.7 8.3zM376 96a40 40 0 1 1 0 80 40 40 0 1 1 0-80z"/></svg>)?) <h0br>

- Model: a function `$f$` that `$f(\text{input})\approx\text{output}$`.

- For example: `$f($` <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#5BC5F7;overflow:visible;position:relative;"><path d="M64 112c-8.8 0-16 7.2-16 16v22.1L220.5 291.7c20.7 17 50.4 17 71.1 0L464 150.1V128c0-8.8-7.2-16-16-16H64zM48 212.2V384c0 8.8 7.2 16 16 16H448c8.8 0 16-7.2 16-16V212.2L322 328.8c-38.4 31.5-93.7 31.5-132 0L48 212.2zM0 128C0 92.7 28.7 64 64 64H448c35.3 0 64 28.7 64 64V384c0 35.3-28.7 64-64 64H64c-35.3 0-64-28.7-64-64V128z"/></svg> `${}_i)\approx \text{type}_i, \text{ for almost every }i$`

--

- More examples:
  - Our jean model: `$\text{waist}=a\times\text{nick}+b$`, where `$a,b$` are the keys.
--

- Exponential decay low (N. of nuclei): `$N(t)=N_0e^{-\lambda t}$`, `$\lambda$` is the key.
--

- [Jim Rohn](https://www.jimrohn.com/): You `$\approx$` average( `$5$` people you spend the most time with ).

---
## I. Some Elements of Machine Learning<h0br> 
### &nbsp; 4. Loss (<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#6EB741;overflow:visible;position:relative;"><path d="M464 256A208 208 0 1 0 48 256a208 208 0 1 0 416 0zM0 256a256 256 0 1 1 512 0A256 256 0 1 1 0 256zm306.7 69.1L162.4 380.6c-19.4 7.5-38.5-11.6-31-31l55.5-144.3c3.3-8.5 9.9-15.1 18.4-18.4l144.3-55.5c19.4-7.5 38.5 11.6 31 31L325.1 306.7c-3.2 8.5-9.9 15.1-18.4 18.4zM288 256a32 32 0 1 0 -64 0 32 32 0 1 0 64 0z"/></svg>?)<h0br>

- What does ' `$\approx$` ' mean?

- .stress[Loss function] quantifies the quality of the model.

- Regression loss: `$\hat{y}=f(\text{input})$` and `$y=$` output ( `$\in\mathbb{R}$` ),

- Quadratic: `$\ell_2(y,\hat{y})=(y-\hat{y})^2$`
  
  - Absolute: `$\ell_1(y,\hat{y})=|y-\hat{y}|$`
  
  - Relative: `$R\ell_1(y,\hat{y})=\Big|\frac{y-\hat{y}}{y}\Big|$` 
]

- `$0\text{-}1$` loss: `$y,\hat{y}\in\{1,...,M\}$`, `$$\ell_{0,1}(y,\hat{y})=\mathbb{1}_{\{y\neq \hat{y}\}}\in\{0,1\}$$`

- Cross-entropy: `$p,\hat{p}\in \mathcal{S}_{M-1}$`, `$$\text{CEn}(p,\hat{p})=-\sum_{j=1}^Mp_j\log(\hat{p}_j)$$`

- KL divergence: `$p,\hat{p}\in \mathcal{S}_{M-1}$`, `$$\text{KL}(p,\hat{p})=\sum_{j=1}^Mp_j\log(p_j/\hat{p}_j)$$`

<h0br>

]

.center[ <h0br>
<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:rgb(233, 165, 9);;overflow:visible;position:relative;"><path d="M336 352c97.2 0 176-78.8 176-176S433.2 0 336 0S160 78.8 160 176c0 18.7 2.9 36.8 8.3 53.7L7 391c-4.5 4.5-7 10.6-7 17v80c0 13.3 10.7 24 24 24h80c13.3 0 24-10.7 24-24V448h40c13.3 0 24-10.7 24-24V384h40c6.4 0 12.5-2.5 17-7l33.3-33.3c16.9 5.4 35 8.3 53.7 8.3zM376 96a40 40 0 1 1 0 80 40 40 0 1 1 0-80z"/></svg> .stress[Hopefully, small (average) loss, good model!]
]

---
## I. Some Elements of Machine Learning<h0br>
### &nbsp; 5. Learning (<svg aria-hidden="true" role="img" viewBox="0 0 320 512" style="height:1em;width:0.62em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#6EB741;overflow:visible;position:relative;"><path d="M160 48a48 48 0 1 1 96 0 48 48 0 1 1 -96 0zM126.5 199.3c-1 .4-1.9 .8-2.9 1.2l-8 3.5c-16.4 7.3-29 21.2-34.7 38.2l-2.6 7.8c-5.6 16.8-23.7 25.8-40.5 20.2s-25.8-23.7-20.2-40.5l2.6-7.8c11.4-34.1 36.6-61.9 69.4-76.5l8-3.5c20.8-9.2 43.3-14 66.1-14c44.6 0 84.8 26.8 101.9 67.9L281 232.7l21.4 10.7c15.8 7.9 22.2 27.1 14.3 42.9s-27.1 22.2-42.9 14.3L247 287.3c-10.3-5.2-18.4-13.8-22.8-24.5l-9.6-23-19.3 65.5 49.5 54c5.4 5.9 9.2 13 11.2 20.8l23 92.1c4.3 17.1-6.1 34.5-23.3 38.8s-34.5-6.1-38.8-23.3l-22-88.1-70.7-77.1c-14.8-16.1-20.3-38.6-14.7-59.7l16.9-63.5zM68.7 398l25-62.4c2.1 3 4.5 5.8 7 8.6l40.7 44.4-14.5 36.2c-2.4 6-6 11.5-10.6 16.1L54.6 502.6c-12.5 12.5-32.8 12.5-45.3 0s-12.5-32.8 0-45.3L68.7 398z"/></svg> <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#5B8D3C;overflow:visible;position:relative;"><path d="M502.6 278.6c12.5-12.5 12.5-32.8 0-45.3l-128-128c-12.5-12.5-32.8-12.5-45.3 0s-12.5 32.8 0 45.3L402.7 224 32 224c-17.7 0-32 14.3-32 32s14.3 32 32 32l370.7 0-73.4 73.4c-12.5 12.5-12.5 32.8 0 45.3s32.8 12.5 45.3 0l128-128z"/></svg> <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:rgb(233, 165, 9);;overflow:visible;position:relative;"><path d="M336 352c97.2 0 176-78.8 176-176S433.2 0 336 0S160 78.8 160 176c0 18.7 2.9 36.8 8.3 53.7L7 391c-4.5 4.5-7 10.6-7 17v80c0 13.3 10.7 24 24 24h80c13.3 0 24-10.7 24-24V448h40c13.3 0 24-10.7 24-24V384h40c6.4 0 12.5-2.5 17-7l33.3-33.3c16.9 5.4 35 8.3 53.7 8.3zM376 96a40 40 0 1 1 0 80 40 40 0 1 1 0-80z"/></svg>?)<hbr>
--

- Learning `$f^*\in\mathcal{F}=\{f:\text{Input space }\mathcal{X}\to\text{ Output space }\mathcal{Y}\}$` means
`$$f^*=\arg\min_{f\in\mathcal{F}}\mathbb{E}[\ell(Y,f(X))]\ \ \ \ \ \ (1)$$`
where,

- `$\ell$`: some loss function.
  - `$(X,Y)\in\mathcal{X}\times\mathcal{Y}$`: generic input-output couple.
  - `$\mathbb{E}$`: expectation w.r.t `$(X,Y)$`.

- Solving `$(1)$` is normally .red[NOT] analytically possible!

- Optimization algorithm: Gradient descent (GD), Stochastic GD, Adagrad, RMSProp...

- Learning: optimization `$\Rightarrow$` best key `$\Rightarrow$` best model (hopefully).

- Ex, jean model: `$(a^*,b^*)=\arg\min_{(a,b)\in\mathbb{R}^2}\mathbb{E}[(\text{waist}-(a\times\text{nick}+b))^2]$`

---
count:false
## I. Some Elements of Machine Learning<h0br>
### &nbsp; 5. Learning (<svg aria-hidden="true" role="img" viewBox="0 0 320 512" style="height:1em;width:0.62em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#6EB741;overflow:visible;position:relative;"><path d="M160 48a48 48 0 1 1 96 0 48 48 0 1 1 -96 0zM126.5 199.3c-1 .4-1.9 .8-2.9 1.2l-8 3.5c-16.4 7.3-29 21.2-34.7 38.2l-2.6 7.8c-5.6 16.8-23.7 25.8-40.5 20.2s-25.8-23.7-20.2-40.5l2.6-7.8c11.4-34.1 36.6-61.9 69.4-76.5l8-3.5c20.8-9.2 43.3-14 66.1-14c44.6 0 84.8 26.8 101.9 67.9L281 232.7l21.4 10.7c15.8 7.9 22.2 27.1 14.3 42.9s-27.1 22.2-42.9 14.3L247 287.3c-10.3-5.2-18.4-13.8-22.8-24.5l-9.6-23-19.3 65.5 49.5 54c5.4 5.9 9.2 13 11.2 20.8l23 92.1c4.3 17.1-6.1 34.5-23.3 38.8s-34.5-6.1-38.8-23.3l-22-88.1-70.7-77.1c-14.8-16.1-20.3-38.6-14.7-59.7l16.9-63.5zM68.7 398l25-62.4c2.1 3 4.5 5.8 7 8.6l40.7 44.4-14.5 36.2c-2.4 6-6 11.5-10.6 16.1L54.6 502.6c-12.5 12.5-32.8 12.5-45.3 0s-12.5-32.8 0-45.3L68.7 398z"/></svg> <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#5B8D3C;overflow:visible;position:relative;"><path d="M502.6 278.6c12.5-12.5 12.5-32.8 0-45.3l-128-128c-12.5-12.5-32.8-12.5-45.3 0s-12.5 32.8 0 45.3L402.7 224 32 224c-17.7 0-32 14.3-32 32s14.3 32 32 32l370.7 0-73.4 73.4c-12.5 12.5-12.5 32.8 0 45.3s32.8 12.5 45.3 0l128-128z"/></svg> <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:rgb(233, 165, 9);;overflow:visible;position:relative;"><path d="M336 352c97.2 0 176-78.8 176-176S433.2 0 336 0S160 78.8 160 176c0 18.7 2.9 36.8 8.3 53.7L7 391c-4.5 4.5-7 10.6-7 17v80c0 13.3 10.7 24 24 24h80c13.3 0 24-10.7 24-24V448h40c13.3 0 24-10.7 24-24V384h40c6.4 0 12.5-2.5 17-7l33.3-33.3c16.9 5.4 35 8.3 53.7 8.3zM376 96a40 40 0 1 1 0 80 40 40 0 1 1 0-80z"/></svg>?)<h0br>
.pull-left[ <h0br>

- Misclassification error of `$f$`:

$$
`\begin{aligned}
\mathbb{E}[\ell_{0,1}(Y,f(X))]&=\mathbb{E}[\mathbb{1}_{\{Y\neq f(X)\}}]\\
&=\mathbb{P}(Y\neq f(X))\\
&=1-\mathbb{P}(Y = f(X))
\end{aligned}`
$$
]
.pull-right[
<br>
<br>
<br>
.center[.stress[Misclassification error = 1 - Accuracy]]
<br>
<bR>
]

.pull-left[ <h0br>
<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-114dd84702271f0b6481" style="width:432px;height:306px;"></div>
<script type="application/json" data-for="htmlwidget-114dd84702271f0b6481">{"x":{"visdat":{"ee860dd7f77":["function () ","plotlyVisDat"]},"cur_data":"ee860dd7f77","attrs":{"ee860dd7f77":{"x":{},"y":{},"showscale":false,"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"z":{},"type":"surface","opacity":0.90000000000000002,"colorscale":[[0,1],["#6AA953","#4FF112"]],"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#06FEFB"}},"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"aspectmode":"manual","aspectratio":{"x":1,"y":1,"z":1},"camera":{"eye":{"x":-1,"y":-1.5,"z":1.8}},"zaxis":{"range":[0,4],"title":"cross_entropy"},"legendgrouptitle":{"text":"f(x,y)"},"xaxis":{"title":"p_true"},"yaxis":{"title":"p_pred"}},"hovermode":"closest","showlegend":false},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false},"data":[{"colorbar":{"title":"cross_entropy","ticklen":2},"colorscale":[[0,"#6AA953"],[1,"#4FF112"]],"showscale":false,"x":[0,0.034482758620689655,0.068965517241379309,0.10344827586206896,0.13793103448275862,0.17241379310344829,0.20689655172413793,0.24137931034482757,0.27586206896551724,0.31034482758620691,0.34482758620689657,0.37931034482758619,0.41379310344827586,0.44827586206896552,0.48275862068965514,0.51724137931034486,0.55172413793103448,0.58620689655172409,0.62068965517241381,0.65517241379310343,0.68965517241379315,0.72413793103448276,0.75862068965517238,0.7931034482758621,0.82758620689655171,0.86206896551724133,0.89655172413793105,0.93103448275862066,0.96551724137931028,1],"y":[0,0.034482758620689655,0.068965517241379309,0.10344827586206896,0.13793103448275862,0.17241379310344829,0.20689655172413793,0.24137931034482757,0.27586206896551724,0.31034482758620691,0.34482758620689657,0.37931034482758619,0.41379310344827586,0.44827586206896552,0.48275862068965514,0.51724137931034486,0.55172413793103448,0.58620689655172409,0.62068965517241381,0.65517241379310343,0.68965517241379315,0.72413793103448276,0.75862068965517238,0.7931034482758621,0.82758620689655171,0.86206896551724133,0.89655172413793105,0.93103448275862066,0.96551724137931028,1],"z":[[0,0.035091319811270061,0.071458963982144977,0.10919929196499197,0.14842000511827333,0.18924199963852842,0.2318016140573243,0.27625337662815824,0.32277339226305102,0.37156355643248296,0.4228568508200336,0.47692407209030929,0.53408248593025787,0.59470710774669278,0.65924562888426386,0.72823850037121551,0.80234647252493729,0.88238918019847357,0.96940055718810358,1.0647107369924282,1.1700712526502548,1.2878542883066382,1.4213856809311605,1.5755363607584192,1.7578579175523736,1.9810014688665831,2.2686835413183646,2.6741486494265283,3.3672958299864719,4],[4,0.14999492361041503,0.16120688416988233,0.18366426608062547,0.21161246938545647,0.2433322037045231,0.27813729497805173,0.31574069746619282,0.35605204385076095,0.39909830457792339,0.44498974344666792,0.49390601985230226,0.54609306159467907,0.60186708584249438,0.66162469341829666,0.72585943583718271,0.79518649442913569,0.87037860453405236,0.95241860942611078,1.042577844365794,1.1425365045048146,1.2545756367189282,1.3818983600931261,1.5292006798376918,1.7037677134863791,1.9178090045994003,2.1942185672027308,2.5844007292387912,3.252392226187327,4],[4,0.26489852740955999,0.25095480435761969,0.25812924019625899,0.27480493365263958,0.29742240777051776,0.3244729758987791,0.3552280183042274,0.38933069543847082,0.42663305272336377,0.46712263607330218,0.51088796761429511,0.55810363725910028,0.60902706393829587,0.66400375795232947,0.72348037130314991,0.7880265163333342,0.85836802886963104,0.93543666166411776,1.0204449517391596,1.1150017563593739,1.2212969851312183,1.3424110392550914,1.4828649989169642,1.6496775094203842,1.8546165403322168,2.1197535930870974,2.4946528090510536,3.1374886223881822,4],[4,0.37980213120870498,0.34070272454535711,0.33259421431189251,0.33799739791982264,0.35151261183651239,0.37080865681950648,0.39471533914226198,0.42260934702618069,0.45416780086880415,0.48925552869993649,0.52786991537628802,0.5701142129235216,0.61618704203409747,0.66638282248636227,0.7211013067691171,0.78086653823753271,0.84635745320520994,0.91845471390212496,0.99831205911252541,1.0874670082139337,1.1880183335435084,1.3029237184170568,1.4365293179962371,1.5955873053543896,1.791424076065034,2.0452886189714636,2.4049048888633164,3.0225850185890373,4],[4,0.49470573500784992,0.43045064473309441,0.40705918842752603,0.40118986218700575,0.40560281590250702,0.41714433774023385,0.4342026599802965,0.45588799861389062,0.48170254901424447,0.51138842132657081,0.54485186313828082,0.58212478858794281,0.62334702012989884,0.66876188702039507,0.71872224223508419,0.77370656014173111,0.83434687754078862,0.90147276614013194,0.97617916648589098,1.0599322600684933,1.1547396819557987,1.2634363975790222,1.3901936370755095,1.5414971012883949,1.7282316117978505,1.9708236448558305,2.3151569686755789,2.9076814147898919,4],[4,0.60960933880699486,0.52019856492083183,0.48152416254315966,0.46438232645418887,0.45969301996850176,0.46348001866096133,0.47368998081833108,0.48916665020160055,0.50923729715968502,0.53352131395320512,0.56183381090027384,0.59413536425236402,0.63050699822570044,0.67114095155442799,0.71634317770105138,0.76654658204592963,0.82233630187636741,0.88449081837813903,0.95404627385925678,1.0323975119230528,1.1214610303680888,1.2239490767409875,1.3438579561547821,1.4874068972224002,1.6650391475306676,1.8963586707401967,2.2254090484878413,2.7927778109907475,4],[4,0.72451294260613985,0.60994648510856919,0.55598913665879313,0.52757479072137192,0.51378322403449639,0.50981569958168871,0.51317730165636566,0.52244530178931048,0.53677204530512534,0.55565420657983933,0.57881575866226675,0.60614593991678534,0.63766697632150204,0.67352001608846079,0.71396411316701858,0.75938660395012814,0.81032572621194621,0.86750887061614612,0.93191338123262246,1.0048627637776124,1.0881823787803788,1.1844617559029531,1.2975222752340547,1.4333166931564056,1.6018466832634846,1.8218936966245634,2.1356611283001041,2.6778742071916031,4],[4,0.83941654640528474,0.69969440529630644,0.63045411077442659,0.59076725498855498,0.56787342810049102,0.55615138050241608,0.55266462249440018,0.5557239533770203,0.56430679345056567,0.57778709920647364,0.59579770642425967,0.61815651558120643,0.64482695441730353,0.6758990806224936,0.71158504863298577,0.75222662585432654,0.79831515054752489,0.85052692285415321,0.90978048860598804,0.97732801563217186,1.0549037271926689,1.1449744350649185,1.2511865943133273,1.3792264890904109,1.5386542189963013,1.7474287225089298,2.045913208112367,2.5629706033924577,4],[4,0.95432015020442973,0.78944232548404392,0.70491908489006005,0.65395971925573826,0.62196363216648565,0.60248706142314346,0.59215194333243482,0.58900260496473023,0.5918415415960061,0.59991999183310796,0.61277965418625258,0.63016709124562775,0.65198693251310502,0.67827814515652629,0.70920598409895297,0.74506664775852505,0.78630457488310379,0.8335449750921603,0.88764759597935383,0.94979326748673154,1.021625075604959,1.1054871142268841,1.2048509133926,1.3251362850244164,1.4754617547291184,1.6729637483932962,1.9561652879246294,2.4480669995933129,4],[4,1.0692237540035747,0.87919024567178128,0.77938405900569374,0.7171521835229212,0.67605383623248039,0.64882274234387094,0.63163926417046934,0.62228125655244015,0.61937628974144654,0.62205288445974227,0.62976160194824549,0.64217766691004896,0.65914691060890662,0.68065720969055921,0.70682691956492016,0.73790666966272356,0.77429399921868258,0.8165630273301675,0.86551470335271952,0.92225851934129111,0.98834642401724915,1.0659997933888494,1.1585152324718726,1.2710460809584219,1.4122692904619354,1.5984987742776628,1.8664173677368923,2.333163395794168,4],[4,1.1841273578027198,0.96893816585951875,0.85384903312132732,0.78034464779010437,0.73014404029847513,0.69515842326459831,0.67112658500850397,0.65555990814015008,0.64691103788688697,0.64418577708637659,0.6467435497102384,0.65418824257447028,0.66630688870470811,0.68303627422459201,0.70444785503088736,0.73074669156692185,0.76228342355426126,0.79958107956817448,0.8433818107260852,0.89472377119585067,0.95506777242953911,1.0265124725508148,1.112179551551145,1.2169558768924271,1.3490768261947521,1.5240338001620293,1.7766694475491547,2.2182597919950231,4],[4,1.2990309616018647,1.058686086047256,0.92831400723696067,0.84353711205728743,0.78423424436446976,0.74149410418532569,0.7106139058465385,0.6888385597278599,0.67444578603232741,0.66631866971301079,0.66372549747223131,0.66619881823889138,0.67346686680050971,0.68541533875862481,0.70206879049685456,0.72358671347112036,0.75027284788984006,0.78259913180618168,0.82124891809945089,0.86718902305041035,0.92178912084182929,0.98702515171278038,1.0658438706304176,1.1628656728264324,1.2858843619275691,1.4495688260463957,1.6869215273614173,2.1033561881958782,4],[4,1.4139345654010098,1.1484340062349934,1.0027789813525942,0.9067295763244706,0.83832444843046439,0.78782978510605317,0.75010122668457302,0.72211721131556994,0.70198053417776785,0.68845156233964522,0.68070744523422422,0.6782093939033127,0.68062684489631131,0.68779440329265773,0.69968972596282175,0.71642673537531887,0.73826227222541885,0.76561718404418877,0.79911602547281668,0.83965427490496991,0.88851046925411947,0.94753783087474586,1.0195081897096905,1.1087754687604379,1.222691897660386,1.3751038519307623,1.5971736071736802,1.9884525843967333,4],[4,1.5288381692001547,1.2381819264227309,1.0772439554682278,0.96992204059165377,0.89241465249645902,0.83416546602678054,0.78958854752260765,0.75539586290327976,0.72951528232320828,0.71058445496627942,0.69768939299621713,0.69021996956773402,0.68778682299211269,0.69017346782669042,0.69731066142878895,0.70926675727951727,0.72625169656099753,0.74863523628219586,0.77698313284618226,0.81211952675952936,0.85523181766640943,0.90805051003671122,0.97317250878896278,1.0546852646944429,1.1594994333932029,1.3006388778151288,1.5074256869859426,1.8735489805975885,4],[4,1.6437417729992996,1.3279298466104681,1.1517089295838612,1.0331145048588368,0.94650485656245364,0.88050114694750781,0.82907586836064218,0.78867451449098958,0.75705003046864849,0.73271734759291374,0.71467134075820993,0.70223054523215511,0.69494680108791429,0.69255253236072323,0.69493159689475614,0.70210677918371578,0.71424112089657643,0.73165328852020295,0.75485024021954816,0.78458477861408915,0.82195316607869962,0.8685631891986767,0.92683682786823551,1.0005950606284484,1.0963069691260199,1.226173903699495,1.4176777667982055,1.7586453767984436,4],[4,1.7586453767984447,1.4176777667982057,1.2261739036994947,1.0963069691260199,1.0005950606284484,0.9268368278682354,0.86856318919867681,0.82195316607869962,0.78458477861408904,0.75485024021954816,0.73165328852020295,0.71424112089657643,0.70210677918371578,0.69493159689475614,0.69255253236072323,0.69494680108791429,0.70223054523215511,0.71467134075820993,0.73271734759291374,0.7570500304686486,0.78867451449098958,0.82907586836064207,0.88050114694750792,0.94650485656245364,1.0331145048588366,1.1517089295838614,1.3279298466104679,1.6437417729992985,4],[4,1.8735489805975896,1.5074256869859428,1.3006388778151283,1.1594994333932032,1.0546852646944429,0.97317250878896266,0.90805051003671133,0.85523181766640943,0.81211952675952936,0.77698313284618226,0.74863523628219586,0.72625169656099753,0.70926675727951727,0.69731066142878895,0.69017346782669042,0.68778682299211269,0.69021996956773402,0.69768939299621713,0.71058445496627942,0.72951528232320828,0.75539586290327976,0.78958854752260743,0.83416546602678066,0.89241465249645902,0.96992204059165366,1.0772439554682278,1.2381819264227307,1.5288381692001536,4],[4,1.9884525843967342,1.5971736071736802,1.3751038519307619,1.222691897660386,1.1087754687604376,1.01950818970969,0.94753783087474575,0.88851046925411936,0.8396542749049698,0.79911602547281657,0.76561718404418866,0.73826227222541885,0.71642673537531887,0.69968972596282164,0.68779440329265773,0.6806268448963112,0.6782093939033127,0.68070744523422422,0.68845156233964522,0.70198053417776785,0.72211721131556983,0.75010122668457302,0.78782978510605328,0.83832444843046439,0.9067295763244706,1.0027789813525945,1.1484340062349934,1.4139345654010089,4],[4,2.1033561881958796,1.6869215273614175,1.4495688260463955,1.2858843619275691,1.1628656728264324,1.0658438706304176,0.98702515171278038,0.92178912084182929,0.86718902305041023,0.82124891809945089,0.78259913180618168,0.75027284788984017,0.72358671347112036,0.70206879049685456,0.68541533875862481,0.67346686680050971,0.66619881823889138,0.66372549747223131,0.66631866971301079,0.67444578603232741,0.6888385597278599,0.71061390584653839,0.7414941041853258,0.78423424436446976,0.84353711205728732,0.92831400723696089,1.0586860860472558,1.2990309616018638,4],[4,2.2182597919950244,1.7766694475491549,1.5240338001620288,1.3490768261947523,1.2169558768924271,1.1121795515511448,1.026512472550815,0.95506777242953911,0.89472377119585056,0.8433818107260852,0.79958107956817448,0.76228342355426137,0.73074669156692185,0.70444785503088736,0.68303627422459201,0.66630688870470811,0.65418824257447028,0.6467435497102384,0.64418577708637659,0.64691103788688709,0.65555990814015008,0.67112658500850386,0.69515842326459842,0.73014404029847513,0.78034464779010437,0.85384903312132743,0.96893816585951864,1.1841273578027192,4],[4,2.3331633957941693,1.8664173677368925,1.5984987742776626,1.4122692904619354,1.2710460809584219,1.1585152324718724,1.0659997933888496,0.98834642401724915,0.92225851934129099,0.86551470335271952,0.8165630273301675,0.77429399921868258,0.73790666966272356,0.70682691956492016,0.68065720969055921,0.65914691060890651,0.64217766691004896,0.62976160194824549,0.62205288445974216,0.61937628974144654,0.62228125655244004,0.63163926417046923,0.64882274234387094,0.67605383623248028,0.71715218352292109,0.77938405900569374,0.87919024567178106,1.0692237540035741,4],[4,2.4480669995933142,1.9561652879246296,1.672963748393296,1.4754617547291187,1.3251362850244164,1.2048509133925998,1.1054871142268841,1.021625075604959,0.94979326748673132,0.88764759597935383,0.8335449750921603,0.78630457488310379,0.74506664775852505,0.70920598409895297,0.67827814515652629,0.65198693251310502,0.63016709124562775,0.61277965418625258,0.59991999183310796,0.59184154159600622,0.58900260496473023,0.5921519433324347,0.60248706142314357,0.62196363216648565,0.65395971925573804,0.70491908489006028,0.78944232548404381,0.95432015020442928,4],[4,2.5629706033924595,2.0459132081123674,1.7474287225089293,1.5386542189963015,1.3792264890904111,1.2511865943133271,1.1449744350649187,1.0549037271926689,0.97732801563217175,0.90978048860598815,0.85052692285415321,0.798315150547525,0.75222662585432654,0.71158504863298577,0.6758990806224936,0.64482695441730353,0.61815651558120654,0.59579770642425967,0.57778709920647375,0.56430679345056589,0.55572395337702041,0.55266462249440018,0.55615138050241619,0.56787342810049113,0.59076725498855509,0.63045411077442681,0.69969440529630655,0.83941654640528451,4],[4,2.6778742071916048,2.1356611283001046,1.8218936966245629,1.6018466832634848,1.4333166931564056,1.2975222752340545,1.1844617559029533,1.0881823787803788,1.0048627637776122,0.93191338123262246,0.86750887061614612,0.81032572621194632,0.75938660395012803,0.71396411316701858,0.67352001608846079,0.63766697632150193,0.60614593991678534,0.57881575866226675,0.55565420657983933,0.53677204530512546,0.52244530178931048,0.51317730165636555,0.50981569958168871,0.51378322403449639,0.52757479072137181,0.55598913665879313,0.60994648510856908,0.72451294260613941,4],[4,2.7927778109907493,2.2254090484878417,1.8963586707401965,1.6650391475306678,1.4874068972224002,1.3438579561547819,1.2239490767409877,1.1214610303680888,1.0323975119230526,0.95404627385925678,0.88449081837813903,0.82233630187636753,0.76654658204592963,0.71634317770105138,0.67114095155442799,0.63050699822570044,0.59413536425236413,0.56183381090027384,0.53352131395320512,0.50923729715968502,0.48916665020160055,0.47368998081833102,0.46348001866096139,0.45969301996850176,0.46438232645418887,0.48152416254315966,0.52019856492083183,0.60960933880699464,4],[4,2.9076814147898942,2.3151569686755793,1.9708236448558301,1.7282316117978507,1.5414971012883949,1.3901936370755092,1.2634363975790224,1.1547396819557987,1.0599322600684931,0.97617916648589098,0.90147276614013194,0.83434687754078873,0.77370656014173111,0.7187222422350843,0.66876188702039518,0.62334702012989895,0.58212478858794292,0.54485186313828093,0.51138842132657092,0.48170254901424459,0.45588799861389062,0.4342026599802965,0.41714433774023402,0.40560281590250713,0.40118986218700581,0.40705918842752625,0.43045064473309452,0.49470573500784992,4],[4,3.0225850185890395,2.4049048888633169,2.0452886189714636,1.7914240760650342,1.5955873053543896,1.4365293179962368,1.302923718417057,1.1880183335435084,1.0874670082139335,0.9983120591125253,0.91845471390212496,0.84635745320521005,0.78086653823753271,0.7211013067691171,0.66638282248636227,0.61618704203409735,0.5701142129235216,0.52786991537628802,0.48925552869993649,0.45416780086880415,0.42260934702618069,0.39471533914226187,0.37080865681950648,0.35151261183651239,0.33799739791982258,0.33259421431189251,0.34070272454535699,0.37980213120870482,4],[4,3.1374886223881844,2.4946528090510536,2.119753593087097,1.854616540332217,1.6496775094203842,1.4828649989169642,1.3424110392550916,1.2212969851312183,1.1150017563593737,1.0204449517391596,0.93543666166411776,0.85836802886963115,0.7880265163333342,0.72348037130314991,0.66400375795232947,0.60902706393829587,0.5581036372591005,0.51088796761429511,0.46712263607330223,0.42663305272336383,0.38933069543847088,0.3552280183042274,0.32447297589877916,0.29742240777051782,0.27480493365263958,0.2581292401962591,0.25095480435761974,0.26489852740956005,4],[4,3.2523922261873288,2.5844007292387912,2.1942185672027303,1.9178090045994003,1.7037677134863789,1.5292006798376914,1.3818983600931261,1.2545756367189282,1.1425365045048141,1.0425778443657938,0.95241860942611067,0.87037860453405236,0.7951864944291358,0.7258594358371826,0.66162469341829677,0.60186708584249426,0.5460930615946793,0.49390601985230226,0.44498974344666792,0.39909830457792339,0.356052043850761,0.31574069746619288,0.27813729497805179,0.24333220370452319,0.21161246938545655,0.18366426608062564,0.16120688416988252,0.1499949236104153,4],[4,3.3672958299864741,2.6741486494265287,2.2686835413183641,1.9810014688665833,1.7578579175523736,1.575536360758419,1.4213856809311607,1.2878542883066382,1.1700712526502546,1.0647107369924282,0.96940055718810358,0.88238918019847368,0.80234647252493729,0.72823850037121551,0.65924562888426386,0.59470710774669278,0.53408248593025798,0.47692407209030929,0.4228568508200336,0.37156355643248296,0.32277339226305102,0.27625337662815824,0.2318016140573243,0.18924199963852842,0.14842000511827333,0.10919929196499197,0.071458963982144977,0.035091319811270172,0]],"type":"surface","opacity":0.90000000000000002,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#06FEFB"}},"frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.20000000000000001,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

]

<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-50888dbf62fc89ef9346" style="width:432px;height:306px;"></div>
<script type="application/json" data-for="htmlwidget-50888dbf62fc89ef9346">{"x":{"visdat":{"ee860dd7f77":["function () ","plotlyVisDat"]},"cur_data":"ee860dd7f77","attrs":{"ee860dd7f77":{"x":{},"y":{},"showscale":false,"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"z":{},"type":"surface","opacity":0.80000000000000004,"colorscale":[[0,1],["#E70B12","#FEC4C6"]],"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#43FFDA"}},"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"aspectmode":"manual","aspectratio":{"x":1,"y":1,"z":1},"camera":{"eye":{"x":-0.5,"y":-1.5,"z":2.1499999999999999}},"zaxis":{"range":[0,4],"title":"KL_div"},"legendgrouptitle":{"text":"f(x,y)"},"xaxis":{"title":"p_true"},"yaxis":{"title":"p_pred"}},"hovermode":"closest","showlegend":false},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false},"data":[{"colorbar":{"title":"KL_div","ticklen":2},"colorscale":[[0,"#E70B12"],[1,"#FEC4C6"]],"showscale":false,"x":[0,0.034482758620689655,0.068965517241379309,0.10344827586206896,0.13793103448275862,0.17241379310344829,0.20689655172413793,0.24137931034482757,0.27586206896551724,0.31034482758620691,0.34482758620689657,0.37931034482758619,0.41379310344827586,0.44827586206896552,0.48275862068965514,0.51724137931034486,0.55172413793103448,0.58620689655172409,0.62068965517241381,0.65517241379310343,0.68965517241379315,0.72413793103448276,0.75862068965517238,0.7931034482758621,0.82758620689655171,0.86206896551724133,0.89655172413793105,0.93103448275862066,0.96551724137931028,1],"y":[0,0.034482758620689655,0.068965517241379309,0.10344827586206896,0.13793103448275862,0.17241379310344829,0.20689655172413793,0.24137931034482757,0.27586206896551724,0.31034482758620691,0.34482758620689657,0.37931034482758619,0.41379310344827586,0.44827586206896552,0.48275862068965514,0.51724137931034486,0.55172413793103448,0.58620689655172409,0.62068965517241381,0.65517241379310343,0.68965517241379315,0.72413793103448276,0.75862068965517238,0.7931034482758621,0.82758620689655171,0.86206896551724133,0.89655172413793105,0.93103448275862066,0.96551724137931028,1],"z":[[4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4],[4,0,0.011211960559467299,0.033669342470210445,0.061617545775041443,0.093337280094108077,0.1281423713676367,0.16574577385577779,0.20605712024034592,0.24910338096750836,0.29499481983625286,0.3439110962418872,0.39609813798426408,0.45187216223207938,0.51162976980788166,0.57586451222676771,0.64519157081872069,0.72038368092363736,0.80242368581569579,0.89258292075537904,0.99254158089439959,1.1045807131085132,1.2319034364827111,1.3792057562272768,1.5537727898759641,1.7678140809889853,2.0442236435923156,2.4344058056283759,3.1023973025769118,4],[4,0.013943723051940304,0,0.0071744358386393037,0.023850129295019895,0.046467603412898073,0.073518171541159416,0.10427321394660771,0.13837589108085113,0.17567824836574408,0.21616783171568249,0.25993316325667543,0.3071488329014806,0.35807225958067618,0.41304895359470978,0.47252556694553022,0.53707171197571446,0.60741322451201141,0.68448185730649813,0.76949014738153987,0.86404695200175419,0.97034218077359857,1.0914562348974717,1.2319101945593445,1.3987227050627644,1.603661735974597,1.8687987887294777,2.2436980046934338,2.8865338180305624,4],[4,0.047207916896812474,0.0081085102334645942,0,0.0054031836079301288,0.018918397524619879,0.038214442507613966,0.062121124830369467,0.090015132714288182,0.12157358655691164,0.15666131438804398,0.19527570106439551,0.23751999861162909,0.28359282772220495,0.33378860817446976,0.38850709245722459,0.4482723239256402,0.51376323889331743,0.58586049959023245,0.6657178448006329,0.75487279390204121,0.85542411923161588,0.97032950410516428,1.1039351036843446,1.2629930910424971,1.4588298617531414,1.7126944046595711,2.0723106745514239,2.6899908042771448,4],[4,0.09351587282084417,0.029260782546088659,0.0058693262405202784,0,0.0044129537155012666,0.015954475553228098,0.033012797793290749,0.054698136426884869,0.080512686827238722,0.11019855913956506,0.14366200095127507,0.18093492640093706,0.22215715794289309,0.26757202483338932,0.31753238004807843,0.37251669795472536,0.43315701535378287,0.50028290395312625,0.57498930429888517,0.65874239788148747,0.75354981976879287,0.86224653539201634,0.98900377488850366,1.1403072391013891,1.3270417496108446,1.5696337826688247,1.913967106488573,2.5064915526028861,4],[4,0.1499163188384931,0.06050554495233007,0.021831142574657902,0.0046893064856871058,0,0.0037869986924595755,0.013996960849829321,0.029473630233098791,0.049544277191183261,0.073828293984703364,0.10214079093177209,0.13444234428386226,0.17081397825719868,0.21144793158592623,0.25665015773254962,0.30685356207742787,0.36264328190786566,0.42479779840963727,0.49435325389075502,0.57270449195455109,0.66176801039958699,0.76425605677248576,0.88416493618628034,1.0277138772538983,1.2053461275621657,1.4366656507716948,1.7657160285193396,2.3330847910222459,4],[4,0.21469724302445115,0.10013078552688048,0.046173437077104418,0.017759091139683214,0.0039675244528076803,0,0.0033616020746769504,0.012629602207621771,0.026956345723436637,0.045838506998150619,0.069000059080578047,0.096330240335096629,0.12785127673981334,0.16370431650677209,0.20414841358532987,0.24957090436843943,0.3005100266302575,0.35769317103445741,0.42209768165093375,0.4950470641959237,0.57836667919869011,0.6746460563212644,0.78770657565236601,0.92350099357471693,1.0920309836817959,1.3120779970428746,1.6258454287184154,2.1680585076099144,4],[4,0.28675192391088455,0.14702978280190626,0.077789488280026409,0.038102632494154798,0.015208805606090836,0.0034867580080159,0,0.0030593308826201149,0.011642170956165487,0.02512247671207346,0.043133083929859484,0.065491893086806252,0.092162331922903351,0.12323445812809342,0.15892042613858559,0.19956200335992635,0.24565052805312471,0.29786230035975303,0.35711586611158785,0.42466339313777168,0.50223910469826871,0.59230981257051829,0.69852197181892717,0.82656186659601072,0.9859895965019011,1.1947641000145297,1.4932485856179669,2.0103059808980577,4],[4,0.3653175452396995,0.20043972051931369,0.11591647992532983,0.064957114291008033,0.032961027201755422,0.01348445645841323,0.0031493383677045905,0,0.002838936631275879,0.010917386868377732,0.023777049221522351,0.041164486280897528,0.062984327548374797,0.089275540191796066,0.12020337913422274,0.15606404279379482,0.19730196991837357,0.24454237012743008,0.29864499101462361,0.36079066252200132,0.43262247064022874,0.51648450926215383,0.61584830842786975,0.73613368005968616,0.88645914976438822,1.0839611434285659,1.3671626829598993,1.8590643946285828,4],[4,0.44984746426212818,0.25981395593033474,0.1600077692642472,0.097775893781474665,0.056677546491033848,0.0294464526024244,0.012262974429022799,0.0029049668109936144,0,0.0026765947182957328,0.010385312206798947,0.022801377168602421,0.039770620867460083,0.061280919949112667,0.087450629823473625,0.11853037992127702,0.15491770947723604,0.19718673758872096,0.24613841361127298,0.30288222959984457,0.36897013427580261,0.44662350364740289,0.53913894273042606,0.65166979121697532,0.79289300072048885,0.97912248453621631,1.2470410779954457,1.7137871060527214,4],[4,0.53994158071634324,0.32475238877314216,0.20966325603495073,0.13615887070372779,0.08595826321209854,0.050972646178221725,0.026940807922127386,0.011374131053773495,0.0027252608005103873,0,0.0025577726238618093,0.010002465488093693,0.022121111618331524,0.038850497138215423,0.060262077944510772,0.086560914480545259,0.11809764646788468,0.15539530248179789,0.19919603363970861,0.25053799410947408,0.31088199534316252,0.3823266954644382,0.46799377446476842,0.57277009980605054,0.70489104910837552,0.87984802307565269,1.1324836704627781,1.5740740149086465,4],[4,0.6353054641296334,0.3949605885750247,0.26458850976472936,0.17981161458505612,0.12050874689223845,0.077768606713094379,0.046888408374307189,0.025113062255628593,0.010720288560096103,0.0025931722407794844,0,0.0024733207666600698,0.0097413693282784042,0.021689841286393508,0.038343293024623248,0.05986121599888905,0.086547350417608748,0.11887363433395037,0.15752342062721958,0.20346352557817904,0.25806362336959798,0.32329965424054907,0.40211837315818633,0.49914017535420108,0.62215886445533775,0.78584332857416439,1.023196029889186,1.4396306907236469,4],[4,0.73572517149769712,0.47022461233168067,0.32456958744928155,0.2285201824211579,0.16011505452715169,0.10962039120274047,0.071891832781260323,0.043907817412257244,0.023771140274455149,0.010242168436332522,0.0024980513309115215,0,0.0024174509929986154,0.0095850093893450339,0.021480332059509055,0.038217341472006172,0.060052878322106151,0.087407790140876074,0.12090663156950399,0.16144488100165721,0.21030107535080678,0.26932843697143316,0.34129879580637779,0.43056607485712517,0.5444825037570733,0.69689445802744965,0.91896421327036748,1.3102431904934206,4],[4,0.84105134620804201,0.55039510343061826,0.38945713247611513,0.28213521759954108,0.20462782950434633,0.14637864303466785,0.10180172453049496,0.067609039911167068,0.041728459331095591,0.022797631974166732,0.0099025700041044384,0.0024331465756213255,0,0.0023866448345777336,0.0095238384366762574,0.021479934287404578,0.038464873568884839,0.06084841329008317,0.089196309854069566,0.12433270376741667,0.16744499467429674,0.22026368704459853,0.28538568579685009,0.36689844170233021,0.47171261040109025,0.61285205482301608,0.8196388639938299,1.1857621576054758,4],[4,0.95118924063857635,0.63537731424974486,0.45915639722313795,0.3405619724981136,0.25395232420173042,0.18794861458678458,0.13652333599991895,0.096121982130266348,0.064497498107925266,0.040164815232190509,0.0221188083974867,0.0096780128714318847,0.0023942687271910623,0,0.0023790645340329153,0.0095542468229925515,0.021688588535853204,0.039100756159479721,0.062297707858824936,0.092032246253365924,0.12940063371797639,0.17601065683795347,0.23428429550751229,0.30804252826772516,0.40375443676529665,0.53362137133877174,0.72512523443748222,1.0660928444377205,4],[4,1.0660928444377213,0.72512523443748245,0.53362137133877152,0.40375443676529665,0.30804252826772516,0.23428429550751217,0.17601065683795358,0.12940063371797639,0.092032246253365813,0.062297707858824936,0.039100756159479721,0.021688588535853204,0.0095542468229925515,0.0023790645340329153,0,0.0023942687271910623,0.0096780128714318847,0.0221188083974867,0.040164815232190509,0.064497498107925377,0.096121982130266348,0.13652333599991884,0.18794861458678469,0.25395232420173042,0.34056197249811337,0.45915639722313817,0.63537731424974464,0.95118924063857524,4],[4,1.1857621576054769,0.81963886399383012,0.61285205482301564,0.47171261040109047,0.36689844170233021,0.28538568579684997,0.22026368704459864,0.16744499467429674,0.12433270376741667,0.089196309854069566,0.06084841329008317,0.038464873568884839,0.021479934287404578,0.0095238384366762574,0.0023866448345777336,0,0.0024331465756213255,0.0099025700041044384,0.022797631974166732,0.041728459331095591,0.067609039911167068,0.10180172453049474,0.14637864303466797,0.20462782950434633,0.28213521759954097,0.38945713247611513,0.55039510343061804,0.8410513462080409,4],[4,1.3102431904934215,0.91896421327036748,0.6968944580274492,0.5444825037570733,0.43056607485712495,0.34129879580637734,0.26932843697143305,0.21030107535080667,0.1614448810016571,0.12090663156950388,0.087407790140875963,0.060052878322106151,0.038217341472006172,0.021480332059508944,0.0095850093893450339,0.0024174509929985044,0,0.0024980513309115215,0.010242168436332522,0.023771140274455149,0.043907817412257133,0.071891832781260323,0.10962039120274059,0.16011505452715169,0.2285201824211579,0.32456958744928177,0.47022461233168067,0.73572517149769623,4],[4,1.4396306907236482,1.0231960298891862,0.78584332857416417,0.62215886445533775,0.49914017535420108,0.40211837315818633,0.32329965424054907,0.25806362336959798,0.20346352557817893,0.15752342062721958,0.11887363433395037,0.086547350417608859,0.05986121599888905,0.038343293024623248,0.021689841286393508,0.0097413693282784042,0.0024733207666600698,0,0.0025931722407794844,0.010720288560096103,0.025113062255628593,0.046888408374307078,0.07776860671309449,0.12050874689223845,0.17981161458505601,0.26458850976472958,0.39496058857502447,0.63530546412963251,4],[4,1.5740740149086478,1.1324836704627783,0.87984802307565224,0.70489104910837574,0.57277009980605054,0.4679937744647682,0.38232669546443843,0.31088199534316252,0.25053799410947397,0.19919603363970861,0.15539530248179789,0.11809764646788479,0.086560914480545259,0.060262077944510772,0.038850497138215423,0.022121111618331524,0.010002465488093693,0.0025577726238618093,0,0.0027252608005104983,0.011374131053773495,0.026940807922127274,0.050972646178221837,0.08595826321209854,0.13615887070372779,0.20966325603495084,0.32475238877314205,0.53994158071634257,4],[4,1.7137871060527228,1.2470410779954459,0.97912248453621609,0.79289300072048885,0.65166979121697532,0.53913894273042584,0.44662350364740311,0.36897013427580261,0.30288222959984445,0.24613841361127298,0.19718673758872096,0.15491770947723604,0.11853037992127702,0.087450629823473625,0.061280919949112667,0.039770620867459971,0.022801377168602421,0.010385312206798947,0.0026765947182956218,0,0.0029049668109935034,0.012262974429022688,0.0294464526024244,0.056677546491033737,0.097775893781474554,0.1600077692642472,0.25981395593033452,0.44984746426212752,4],[4,1.8590643946285841,1.3671626829598993,1.0839611434285659,0.88645914976438844,0.73613368005968616,0.61584830842786953,0.51648450926215383,0.43262247064022874,0.36079066252200109,0.29864499101462361,0.24454237012743008,0.19730196991837357,0.15606404279379482,0.12020337913422274,0.089275540191796066,0.062984327548374797,0.041164486280897528,0.023777049221522351,0.010917386868377732,0.0028389366312759901,0,0.0031493383677044795,0.013484456458413341,0.032961027201755422,0.064957114291007811,0.11591647992533005,0.20043972051931358,0.36531754523969906,4],[4,2.0103059808980595,1.4932485856179674,1.1947641000145293,0.98598959650190132,0.82656186659601094,0.69852197181892695,0.59230981257051851,0.50223910469826871,0.42466339313777157,0.35711586611158797,0.29786230035975303,0.24565052805312482,0.19956200335992635,0.15892042613858559,0.12323445812809342,0.092162331922903351,0.065491893086806363,0.043133083929859484,0.025122476712073571,0.011642170956165709,0.0030593308826202259,0,0.003486758008016011,0.015208805606090947,0.038102632494154909,0.077789488280026631,0.14702978280190637,0.28675192391088433,4],[4,2.1680585076099161,1.6258454287184159,1.3120779970428742,1.0920309836817961,0.92350099357471693,0.78770657565236579,0.67464605632126462,0.57836667919869011,0.49504706419592348,0.42209768165093375,0.35769317103445741,0.30051002663025761,0.24957090436843932,0.20414841358532987,0.16370431650677209,0.12785127673981322,0.096330240335096629,0.069000059080578047,0.045838506998150619,0.026956345723436748,0.012629602207621771,0.0033616020746768394,0,0.0039675244528076803,0.017759091139683103,0.046173437077104418,0.10013078552688037,0.2146972430244507,4],[4,2.3330847910222476,1.7657160285193401,1.4366656507716948,1.2053461275621662,1.0277138772538983,0.88416493618628011,0.76425605677248598,0.66176801039958699,0.57270449195455086,0.49435325389075502,0.42479779840963727,0.36264328190786577,0.30685356207742787,0.25665015773254962,0.21144793158592623,0.17081397825719868,0.13444234428386237,0.10214079093177209,0.073828293984703364,0.049544277191183261,0.029473630233098791,0.013996960849829265,0.0037869986924596311,0,0.0046893064856871058,0.021831142574657902,0.06050554495233007,0.14991631883849288,4],[4,2.5064915526028884,1.9139671064885735,1.5696337826688243,1.3270417496108449,1.1403072391013891,0.98900377488850344,0.86224653539201657,0.75354981976879287,0.65874239788148725,0.57498930429888517,0.50028290395312613,0.43315701535378293,0.37251669795472531,0.31753238004807849,0.26757202483338938,0.22215715794289315,0.18093492640093711,0.14366200095127513,0.11019855913956511,0.080512686827238777,0.054698136426884814,0.033012797793290694,0.015954475553228209,0.0044129537155013221,0,0.0058693262405204449,0.029260782546088715,0.093515872820844115,4],[4,2.689990804277147,2.0723106745514244,1.7126944046595711,1.4588298617531417,1.2629930910424971,1.1039351036843443,0.9703295041051645,0.85542411923161588,0.75487279390204098,0.66571784480063279,0.58586049959023245,0.51376323889331754,0.4482723239256402,0.38850709245722459,0.33378860817446976,0.28359282772220484,0.23751999861162909,0.19527570106439551,0.15666131438804398,0.12157358655691164,0.090015132714288182,0.062121124830369356,0.038214442507613966,0.018918397524619879,0.0054031836079300732,0,0.0081085102334644832,0.047207916896812308,4],[4,2.8865338180305646,2.2436980046934338,1.8687987887294772,1.6036617359745973,1.3987227050627644,1.2319101945593445,1.0914562348974719,0.97034218077359857,0.86404695200175397,0.76949014738153987,0.68448185730649802,0.60741322451201141,0.53707171197571446,0.47252556694553016,0.41304895359470972,0.35807225958067612,0.30714883290148076,0.25993316325667537,0.21616783171568249,0.17567824836574408,0.13837589108085113,0.10427321394660766,0.073518171541159416,0.046467603412898073,0.02385012929501984,0.0071744358386393592,0,0.013943723051940304,4],[4,3.1023973025769136,2.4344058056283759,2.0442236435923151,1.7678140809889851,1.5537727898759637,1.3792057562272761,1.2319034364827108,1.104580713108513,0.99254158089439881,0.89258292075537848,0.80242368581569534,0.72038368092363703,0.64519157081872047,0.57586451222676727,0.51162976980788144,0.45187216223207893,0.39609813798426396,0.34391109624188698,0.29499481983625264,0.24910338096750809,0.2060571202403457,0.16574577385577757,0.12814237136763648,0.093337280094107883,0.061617545775041249,0.033669342470210334,0.011211960559467216,0,4],[4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4]],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#43FFDA"}},"frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.20000000000000001,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

]

---
## I. Some Elements of Machine Learning<h0br> 
### &nbsp; 6. Evaluation (how good is the model?)<h0br>

- .stress[Good model] means small (average) loss, but on which data?

- The model should predict fairly well on new observation.

- Model evaluation: computing the loss on new (unseen) observations.

### &nbsp; In practice

- We split the data into `$2$` parts: Training ( `$\approx80\%$` ) & testing ( `$\approx20\%$` ).

- Training data: to build the model (estimate the good key).
  
  - Testing data: to access the performance.
  
---
template: inter-slide
class: left, middle
count: false

<br>

<br>

<br>

<br>

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; `Tips` dataset <h0br>

- Dimension: `$244\times 7$`.

```r
tip[sample(nrow(tip), 7),] |> knitr::kable(format = "markdown")
```

| total_bill|  tip|sex    |smoker |day  |time   | size|
|----------:|----:|:------|:------|:----|:------|----:|
|      16.29| 3.71|Male   |No     |Sun  |Dinner |    3|
|      27.20| 4.00|Male   |No     |Thur |Lunch  |    4|
|      28.15| 3.00|Male   |Yes    |Sat  |Dinner |    5|
|      17.07| 3.00|Female |No     |Sat  |Dinner |    3|
|      21.01| 3.50|Male   |No     |Sun  |Dinner |    3|
|      31.27| 5.00|Male   |No     |Sat  |Dinner |    3|
|      18.35| 2.50|Male   |No     |Sat  |Dinner |    4|

- Objective: build a linear model to predict the .stress[tip].

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; `Tips` dataset <h0br>

- Pearson's correlation coefficient:

`$$\rho(X,Y)=\frac{\sum_{i=1}^n(X_i-\overline{X}_n)(Y_i-\overline{Y}_n)}{\sqrt{\sum_{i=1}^n(X_i-\overline{X}_n)^2}\sqrt{\sum_{i=1}^n(Y_i-\overline{Y}_n)^2}}.$$`
  - `$-1\leq \rho(X,Y)\leq 1$` for any `$X,Y$`.
  
  - `$|\rho(X,Y)|\approx 1\Leftrightarrow Y\approx aX+b$` for some `$a\neq 0$`.
  
- On **Tips** dataset:

|           | total_bill|       tip|      size|
|:----------|----------:|---------:|---------:|
|total_bill |  1.0000000| 0.6757341| 0.5983151|
|tip        |  0.6757341| 1.0000000| 0.4892988|
|size       |  0.5983151| 0.4892988| 1.0000000|

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; `Tips` dataset <h0br>

```r
tip |> ggplot(aes(x = total_bill, y = tip, color = sex, size = size, shape = time)) + geom_point() +
  labs(title = "Tip VS total bill", x = "Total bill", y = "Tip", size = '', color = '', shape = '') +
  theme(rect = element_rect(colour = "white"),
        axis.text.x = element_text(size = 15),
        axis.text.y = element_text(size = 15),
        legend.position="bottom", legend.box = "horizontal")
```

<img src="itc_linear_files/figure-html/tip1-1.png" style="display: block; margin: auto;" />
]

```r
tip |> ggplot(aes(x = day, y = tip, fill = smoker))+
  geom_violin() +
  labs(title = "Tip VS Day and Smoke", x = "Day", y = "Tip") +
  theme(rect = element_rect(colour = "white"),
                 axis.text.x = element_text(size = 15),
                 axis.text.y = element_text(size = 15),
                 legend.position="bottom", legend.box = "horizontal")
```

<img src="itc_linear_files/figure-html/tip2-1.png" style="display: block; margin: auto;" />
]

---

## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Simple LR: `Tip ~ Total_bill` <h0br>

- Simple start: **Total bill** `$\rightarrow$` **Tip**.

- Model: `$\hat{y}=ax +b$`

- Criterion: `$\text{MSE}(a,b)=\frac{1}{n}\sum_{i=1}^n(y_i-\hat{y}_i)^2=\frac{1}{n}\sum_{i=1}^n(y_i-ax_i-b)^2$`

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Simple LR: `Tip ~ Total_bill` <h0br>

- Simple start: **Total bill** `$\rightarrow$` **Tip**.

- Model: `$\hat{y}=ax +b$`

- Criterion: `$\text{MSE}(a,b)=\frac{1}{n}\sum_{i=1}^n(y_i-\hat{y}_i)^2=\frac{1}{n}\sum_{i=1}^n(y_i-ax_i-b)^2$`

<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-616f9beb0beab8268b03" style="width:720px;height:324px;"></div>
<script type="application/json" data-for="htmlwidget-616f9beb0beab8268b03">{"x":{"visdat":{"ee8609d3689":["function () ","plotlyVisDat"]},"cur_data":"ee8609d3689","attrs":{"ee8609d3689":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"camera":{"eye":{"x":0,"y":-1.25,"z":0.69999999999999996}},"xaxis":{"title":"a"},"yaxis":{"title":"b"},"zaxis":{"title":"Loss"}},"hovermode":"closest","showlegend":false,"legend":{"yanchor":"top","y":0.5}},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false},"data":[{"colorbar":{"title":"Loss","ticklen":2,"len":0.5,"lenmode":"fraction","y":1,"yanchor":"top"},"colorscale":[["0","rgba(68,1,84,1)"],["0.0416666666666667","rgba(70,19,97,1)"],["0.0833333333333333","rgba(72,32,111,1)"],["0.125","rgba(71,45,122,1)"],["0.166666666666667","rgba(68,58,128,1)"],["0.208333333333333","rgba(64,70,135,1)"],["0.25","rgba(60,82,138,1)"],["0.291666666666667","rgba(56,93,140,1)"],["0.333333333333333","rgba(49,104,142,1)"],["0.375","rgba(46,114,142,1)"],["0.416666666666667","rgba(42,123,142,1)"],["0.458333333333333","rgba(38,133,141,1)"],["0.5","rgba(37,144,140,1)"],["0.541666666666667","rgba(33,154,138,1)"],["0.583333333333333","rgba(39,164,133,1)"],["0.625","rgba(47,174,127,1)"],["0.666666666666667","rgba(53,183,121,1)"],["0.708333333333333","rgba(79,191,110,1)"],["0.75","rgba(98,199,98,1)"],["0.791666666666667","rgba(119,207,85,1)"],["0.833333333333333","rgba(147,214,70,1)"],["0.875","rgba(172,220,52,1)"],["0.916666666666667","rgba(199,225,42,1)"],["0.958333333333333","rgba(226,228,40,1)"],["1","rgba(253,231,37,1)"]],"showscale":true,"x":[-0.20000000000000001,-0.1736842105263158,-0.14736842105263159,-0.12105263157894738,-0.094736842105263175,-0.068421052631578966,-0.042105263157894757,-0.015789473684210548,0.010526315789473661,0.03684210526315787,0.06315789473684208,0.089473684210526316,0.1157894736842105,0.14210526315789468,0.16842105263157892,0.19473684210526315,0.22105263157894733,0.24736842105263152,0.27368421052631575,0.29999999999999999],"y":[-8,-7.0526315789473681,-6.1052631578947372,-5.1578947368421053,-4.2105263157894743,-3.2631578947368425,-2.3157894736842106,-1.3684210526315796,-0.42105263157894779,0.52631578947368318,1.473684210526315,2.4210526315789469,3.3684210526315788,4.3157894736842106,5.2631578947368407,6.2105263157894726,7.1578947368421044,8.1052631578947363,9.0526315789473664,10],"z":[[232.04548890163935,215.53004656858226,199.66614392661552,184.45378097573908,169.89295771595297,155.98367414725718,142.72593026965171,130.11972608313656,118.16506158771173,106.86193678337725,96.210351670133065,86.210306247979204,76.861800516915679,68.164834476942488,60.11940812805959,52.725521470267019,45.98317450356479,39.892367227952874,34.45309964343128,29.665371750000002],[204.60632110726124,189.07743547008536,174.20008952399982,159.97428326900459,146.40001670509969,133.47728983228509,121.20610265056084,109.58645515992689,98.618347360383268,88.301779251929986,78.63675083456701,69.623262108294355,61.261313073112035,53.550903729020035,46.492034076018349,40.084704114106984,34.32891384328596,29.22466326355525,24.771952374914857,20.970781177364788],[178.96216716329869,164.41983822200402,150.52904897179965,137.28979941268562,124.70208954466193,112.76591936772854,101.48128888188549,90.848198087132758,80.866646983470332,71.536635570898241,62.858163849416478,54.831231819025028,47.455839479723913,40.731986831513119,34.659673874392638,29.238900608362478,24.469667033422652,20.351973149573148,16.885818956813957,14.071204455145091],[155.11302706975161,141.55725482433814,128.65302227001499,116.40032940678216,104.79917623463967,93.849562753587492,83.551488963625644,73.904954864754103,64.909960456972897,56.566505740282011,48.874590714681446,41.834215380171202,35.445379736751292,29.708083784421703,24.622327523182424,20.188110953033469,16.405434073974849,13.274296886006546,10.794699389128562,8.9666415833409019],[133.05890082662006,120.4896852770878,108.57200941864586,97.30587325129423,86.691276775032946,76.728219989861969,67.416702895781313,58.756725492790991,50.748287780890983,43.391389760081303,36.686031430361943,30.632212791732901,25.229933844194189,20.479194587745802,16.379995022387732,12.93233514811998,10.136214964942562,7.9916344728554636,6.4985936718586848,5.6570925619522292],[112.79978843390401,101.21712958025296,90.286010417692211,80.006430946221798,70.378391165841705,61.401891076551941,53.076930678352497,45.403509971243366,38.381628955224571,32.011287630296088,26.292485996457934,21.225224053710097,16.809501802052594,13.04531924148541,9.932676372008542,7.4715731936219996,5.662009706325783,4.503985910119888,3.997501805004315,4.1425573909790652],[94.335689891603479,83.739587733833616,73.795025267154088,64.50200249156488,55.860519407065986,47.870576013657427,40.532172311339181,33.845308300111263,27.809983979973669,22.426199350926396,17.693954412969443,13.613249166102811,10.18408361032651,7.4064577456405312,5.280371572044869,3.8058250895395309,2.9828182981245184,2.8113511977998269,3.2914237885654591,4.4230360704214151],[77.666605199718461,68.057059737829817,59.09905396703148,50.792587887323478,43.137661498705796,36.134274801178435,29.782427794741395,24.082120479394678,19.033352855138286,14.636124921972216,10.890436679896471,7.7962881289110433,5.3536792690159443,3.5626101002111676,2.4230806224967103,1.9350908358725769,2.0986407403387668,2.913730335895278,4.3803596225421151,6.4985286002792755],[62.792534358248957,54.169545592241505,46.19809651732438,38.878187133497583,32.2098174407611,26.192987439114944,20.827697128559109,16.113946509093598,12.051735580718411,8.6410643434335466,5.8819327972390036,3.7743409421347818,2.3182887781208867,1.5137763051973132,1.3608035233640614,1.8593704326211338,3.0094770329685274,4.8111233244062417,7.2643093069342841,10.36903498055265],[49.713477367194969,42.077045297068722,35.092152918032802,28.758800230087203,23.076987233231929,18.046713927466975,13.667980312792345,9.9407863892080375,6.8651321567140524,4.4410176153103906,2.6684427649970521,1.5474076057740354,1.077912137641343,1.2599563605989719,2.0935402746469247,3.5786638797852017,5.7153271760137976,8.5035301633327158,11.943272841741962,16.034555211241532],[38.429434226556481,31.779558852311439,25.781223169156725,20.434427177092328,15.739170876118258,11.69545426623451,8.3032773474410853,5.5626401197379822,3.473542583125202,2.0359847376027451,1.2499665831706106,1.1154881198287994,1.6325493475773105,2.8011502664161427,4.6212908763453013,7.0929711773647837,10.216191169474584,13.990950852674704,18.417250226965155,23.495089292345934],[28.940404936333504,23.277086257969671,18.265307270696159,13.905067974512971,10.196368369420103,7.1392084554175597,4.7335882325053387,2.9795077006834405,1.8769668599518652,1.4259657103106125,1.6265042517596828,2.4785824842990771,3.9822004079287914,6.1373580226488276,8.944055328459191,12.40229232535988,16.512069013350882,21.273385392432207,26.686241462603867,32.750637223865844],[21.246389496526046,16.569627514043415,12.544405222651109,9.1707226223491229,6.4485797131374625,4.3779764950161226,2.9589129679851061,2.1913891320444123,2.0754049871940414,2.6109605334339938,3.7980557707642681,5.636690699184868,8.1268653186957867,11.268579629297026,15.061833630988595,19.506627323770488,24.602960707642694,30.350833782605225,36.75024654865809,43.801199005801273],[15.347387907134101,11.657182620532675,8.6185170250215712,6.231391120600791,4.4958049072703341,3.4117583850301991,2.979251553880387,3.1982844138208981,4.0688569648517321,5.590969206972888,7.7646211401843672,10.589812764486172,14.066544079878295,18.194815086360737,22.974625783933512,28.405976172596613,34.488866252350022,41.223296023193754,48.609265485127821,56.646774638152216],[11.243400168157674,8.5397515774374497,6.4876426778075498,5.0870734692679722,4.3380439518187179,4.2405541254597861,4.7946039901911766,6.0001935460128903,7.8573227929249265,10.365991730927286,13.526200360019967,17.337948680202977,21.801236691476301,26.916064393839946,32.682431787293922,39.100338871838225,46.169785647472835,53.890772114197773,62.263298272013046,71.287364120918639],[8.9344262795967513,7.2173343847577316,6.151782181009037,5.7377696683506629,5.975296846782614,6.8643637163048856,8.4049702769174814,10.597116528620399,13.44080247141364,16.936028105297204,21.082793430271092,25.881098446335304,31.33094315348983,37.43232755173468,44.185251641069861,51.58971542149537,59.645718893011185,68.353262055617321,77.712344909313799,87.722967454100612],[8.4204662414513418,7.6899310424935274,7.6109355346260363,8.1834797178488685,9.4075635921620222,11.2831871575655,13.810350414059299,16.989053361643421,20.819296000317866,25.301078330082635,30.434400350937725,36.219262062883146,42.655663465918877,49.743604560044929,57.483085345261323,65.874105821568037,74.91666598896505,84.610765847452399,94.956405397030082,105.95358463769809],[9.7015200537214472,9.9575415506448373,10.86510273865855,12.424203617762586,14.634844187956945,17.497024449241625,21.01074440161663,25.176004045081957,29.992803379637607,35.461142405283582,41.581021122019877,48.3524395298465,55.775397628763436,63.849895418770693,72.575932899868292,81.953510072056204,91.982626935334423,102.66328348970298,113.99547973516187,125.97921567171107],[12.777587716407057,14.02016590921165,15.914283793106566,18.459941368091805,21.657138634167367,25.50587559133325,30.006152239589458,35.157968578935986,40.961324609372838,47.416220330900018,54.522655743517511,62.280630847225346,70.690145642023481,79.751200127911929,89.46379430489074,99.827928172959858,110.84360173211928,122.51081498236903,134.82956792370913,147.79986055613955],[17.648669229508197,19.877804118193996,22.758478697970119,26.290692968836563,30.474446930793331,35.309740583840423,40.796573927977832,46.934946963205569,53.724859689523626,61.166312106932011,69.259304215430717,78.003836015019758,87.39990750569909,97.447518687468758,108.14666956032876,119.4973601242791,131.49959037931973,144.15336032545068,157.45866996267196,171.41551929098361]],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.20000000000000001,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Simple LR: `Tip ~ Total_bill` <h0br>

- Simple start: **Total bill** `$\rightarrow$` **Tip**.

- Model: `$\hat{y}=ax +b$`

- Criterion: `$\text{MSE}(a,b)=\frac{1}{n}\sum_{i=1}^n(y_i-\hat{y}_i)^2=\frac{1}{n}\sum_{i=1}^n(y_i-ax_i-b)^2$`

```

Call:
lm(formula = tip ~ total_bill, data = tip[mask, ])

Residuals:
    Min      1Q  Median      3Q     Max 
-3.0255 -0.6008 -0.1207  0.4884  3.3558

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 1.127194   0.182384    6.18 3.72e-09 ***
total_bill  0.093461   0.008437   11.08  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.024 on 193 degrees of freedom
Multiple R-squared:  0.3887,	Adjusted R-squared:  0.3855 
F-statistic: 122.7 on 1 and 193 DF,  p-value: < 2.2e-16
```

]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Simple LR: `Tip ~ Total_bill` <h0br>

- Simple start: **Total bill** `$\rightarrow$` **Tip**.

- Model: `$\hat{y}=ax +b$`

- Criterion: `$\text{MSE}(a,b)=\frac{1}{n}\sum_{i=1}^n(y_i-\hat{y}_i)^2=\frac{1}{n}\sum_{i=1}^n(y_i-ax_i-b)^2$`

`$$\hat{a}=\frac{\langle x-\overline{x},y-\overline{y}\rangle}{\|x-\overline{x}\|^2}=\frac{\rho(x,y)}{\mathbb{V}(x)},\hspace{0.3cm}\\ 
\hat{b}=\overline{y}-\hat{a}\overline{x}\hspace{5.3cm}$$`

]
.pull-right[ <hbr>

```r
p1 + geom_point(data =tibble(total_bill = tip[!mask,][['total_bill']], tip = predict(mod0, newdata = tip[!mask,])), color = "#791AD3", shape = "x", size = 4) + 
  geom_abline(slope = mod0$coefficients[2], intercept = mod0$coefficients[1], color = "blue", linewidth = 1)+
  labs(title = "Fitting linear line") +
  annotate('text', x = 30, y = 8.5, label = paste("RMSE =", round(er, 3), "at a =",round(mod0$coefficients[2], 3), "and b =", round(mod0$coefficients[1], 3), "\n R-squared:", round(summary(mod0)$r.squared, 3))) -> p3; p3
```

<img src="itc_linear_files/figure-html/fit1-1.png" style="display: block; margin: auto;" />
]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Simple LR: `Tip ~ Total_bill` <h0br>

- Simple start: **Total bill** `$\rightarrow$` **Tip**.

- Model: `$\hat{y}=ax +b$`

- Criterion: `$\text{MSE}(a,b)=\frac{1}{n}\sum_{i=1}^n(y_i-\hat{y}_i)^2=\frac{1}{n}\sum_{i=1}^n(y_i-ax_i-b)^2$`

`$$\hat{a}=\frac{\langle x-\overline{x},y-\overline{y}\rangle}{\|x-\overline{x}\|^2}=\frac{\rho(x,y)}{\mathbb{V}(x)},\\ 
\hat{b}=\overline{y}-\hat{a}\overline{x}\hspace{5.3cm}$$`

- .stress[R-squared]: `$R^2=\mathbb{V}(\hat{y})/\mathbb{V}(y)$`.
]
.pull-right[ <hbr>

```r
p1 + geom_point(data =tibble(total_bill = tip[!mask,][['total_bill']], tip = predict(mod0, newdata = tip[!mask,])), color = "red", shape = "x", size = 4) + 
  geom_abline(slope = mod0$coefficients[2], intercept = mod0$coefficients[1], color = "blue", linewidth = 1)+
  labs(title = "Fitting linear line") +
  annotate('text', x = 30, y = 8.5, label = paste("RMSE:", round(er, 3), "at a =",round(mod0$coefficients[2], 3), "and b =", round(mod0$coefficients[1], 3))) -> p3; p3
```

![](itc_linear_files/figure-html/fit1-out-1.png)
]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Simple LR: `Tip ~ Total_bill` <h0br>

- Simple start: **Total bill** `$\rightarrow$` **Tip**.

- Model: `$\hat{y}=ax +b$`

- Criterion: `$\text{MSE}(a,b)=\frac{1}{n}\sum_{i=1}^n(y_i-\hat{y}_i)^2=\frac{1}{n}\sum_{i=1}^n(y_i-ax_i-b)^2$`

`$$\hat{a}=\frac{\langle x-\overline{x},y-\overline{y}\rangle}{\|x-\overline{x}\|^2}=\frac{\rho(x,y)}{\mathbb{V}(x)},\\ 
\hat{b}=\overline{y}-\hat{a}\overline{x}\hspace{5.3cm}$$`

- .stress[R-squared]: `$R^2=\mathbb{V}(\hat{y})/\mathbb{V}(y)$`.

- .red[Not possible] in general!
]

![](itc_linear_files/figure-html/fit1-out-1.png)
]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Simple LR: `Tip ~ Total_bill` <h0br>

- Simple start: **Total bill** `$\rightarrow$` **Tip**.

- Model: `$\hat{y}=ax +b$`

- Criterion: `$\text{MSE}(a,b)=\frac{1}{n}\sum_{i=1}^n(y_i-\hat{y}_i)^2=\frac{1}{n}\sum_{i=1}^n(y_i-ax_i-b)^2$`

`$$\hat{a}=\frac{\langle x-\overline{x},y-\overline{y}\rangle}{\|x-\overline{x}\|^2}=\frac{\rho(x,y)}{\mathbb{V}(x)},\\ 
\hat{b}=\overline{y}-\hat{a}\overline{x}\hspace{5.3cm}$$`

- .stress[R-squared]: `$R^2=\mathbb{V}(\hat{y})/\mathbb{V}(y)$`.

- .red[Not possible] in general!

- .stress[Numerical methods] are needed!
]

![](itc_linear_files/figure-html/fit1-out-1.png)
]

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Optimization algorithm: Gradient Descent (GD) <h0br>

- We consider the optimization problem: <hbr> `$$x^*=\arg\min_{x\in O}f(x),$$` <hbr>
where `$f:\mathbb{R}^d\to\mathbb{R}$` is **differentiable** on some open subset `$O\subset\mathbb{R}^d$`.

.center[<img src="./img/min.png" width="400px"/>]
  
---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Optimization algorithm: Gradient Descent (GD) <h0br>

- We consider the optimization problem: <hbr> `$$x^*=\arg\min_{x\in O}f(x),$$` <hbr>
where `$f:\mathbb{R}^d\to\mathbb{R}$` is **differentiable** on some open subset `$O\subset\mathbb{R}^d$`.

- .stress[Key idea]: for any `$x_0, x_0+h\in O$` and `$\alpha>0$`: `$f(x_0+h)\approx f(x_0)+h^t\nabla f(x_0)$`.

.center[<img src="./img/min.png" width="400px"/>]
  
---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Optimization algorithm: Gradient Descent (GD) <h0br>

- We consider the optimization problem: <hbr> `$$x^*=\arg\min_{x\in O}f(x),$$` <hbr>
where `$f:\mathbb{R}^d\to\mathbb{R}$` is **differentiable** on some open subset `$O\subset\mathbb{R}^d$`.

- .stress[Key idea]: for any `$x_0, x_0+h\in O$` and `$\alpha>0$`: `$f(x_0+h)\approx f(x_0)+h^t\nabla f(x_0)$`.

If `$h=-\alpha \nabla f(x_0)$`, thus `$f(x_0-\alpha \nabla f(x_0))\approx f(x_0)-\alpha\|\nabla f(x_0)\|^2\leq f(x_0)$`.

> .stress[Gradient descent algorithm]:
<br>
  - initialize: `$x_0$`, learning rate `$\alpha>0$`, threshold `$\delta>0$`
  - `for t = 0, 1, ..., T:`
      `$$x_{t+1}=x_t-\alpha\nabla f(x_t)$$`
  - stop when `$\|x^{t+1}-x^t\|<\delta$` or other stopping criterion is met.
  
---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Optimization algorithm: Gradient Descent (GD) <h0br>

> .stress[Gradient descent algorithm]:
<br>
  - initialize: `$x_0$`, learning rate `$\alpha>0$`, threshold `$\delta>0$`
  - `for t = 0, 1, ..., T:`
      `$$x_{t+1}=x_t-\alpha\nabla f(x_t)$$`
  - stop when `$\|x^{t+1}-x^t\|<\delta$` or other stopping criterion is met.
  
.pull-left-60[<hbr>
- .stress[Ideal case]: if `$f$` is **convex** & `$\nabla f$` is `$L$`-**Lipschitz continous**, i.e.,
`$$\|\nabla f(x)-\nabla f(y)\|\leq L\|x-y\|,$$`
  thus **GD** algorithm converges `$\Leftrightarrow \alpha< 2/L$`. 
  
]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Optimization algorithm: Gradient Descent (GD) <h0br>

> .stress[Gradient descent algorithm]:
<br>
  - initialize: `$x_0$`, learning rate `$\alpha>0$`, threshold `$\delta>0$`
  - `for t = 0, 1, ..., T:`
      `$$x_{t+1}=x_t-\alpha\nabla f(x_t)$$`
  - stop when `$\|x^{t+1}-x^t\|<\delta$` or other stopping criterion is met.
  
.pull-left-60[<hbr>
- .stress[Ideal case]: if `$f$` is **convex** & `$\nabla f$` is `$L$`-**Lipschitz continous**, i.e.,
`$$\|\nabla f(x)-\nabla f(y)\|\leq L\|x-y\|,$$`
  thus **GD** algorithm converges `$\Leftrightarrow \alpha< 2/L$`.

- Main issue in ML: computing full `$\nabla f$`.

]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Optimization algorithm: Gradient Descent (GD) <h0br>

> .stress[Gradient descent algorithm]:
<br>
  - initialize: `$x_0$`, learning rate `$\alpha>0$`, threshold `$\delta>0$`
  - `for t = 0, 1, ..., T:`
      `$$x_{t+1}=x_t-\alpha\nabla f(x_t)$$`
  - stop when `$\|x^{t+1}-x^t\|<\delta$` or other stopping criterion is met.
  
.pull-left-60[<hbr>
- .stress[Ideal case]: if `$f$` is **convex** & `$\nabla f$` is `$L$`-**Lipschitz continous**, i.e.,
`$$\|\nabla f(x)-\nabla f(y)\|\leq L\|x-y\|,$$`
  thus **GD** algorithm converges `$\Leftrightarrow \alpha< 2/L$`.

- Main issue in ML: computing full `$\nabla f$`.

- A common approach: numerical method using sub-samples (mini-batch).

]

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Optimization algorithm: Gradient Descent (GD) <h0br>

<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-623decd7b1af20295710" style="width:800px;height:450px;"></div>
<script type="application/json" data-for="htmlwidget-623decd7b1af20295710">{"x":{"visdat":{"ee8183a4189":["function () ","plotlyVisDat"]},"cur_data":"ee8183a4189","attrs":{"ee8183a4189":{"x":{},"y":{},"z":{},"showlegend":false,"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"inherit":true},"ee8183a4189.1":{"x":{},"y":{},"z":{},"type":"scatter3d","mode":"lines","showlegend":false,"line":{"color":"#07FAFA","width":5},"inherit":false},"ee8183a4189.2":{"x":{},"y":{},"z":{},"showlegend":false,"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"scatter3d","mode":"markers","frame":{},"marker":{"color":"#70EF44"},"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"aspectmode":"manual","aspectratio":{"x":1,"y":1,"z":2},"legendgrouptitle":{"text":"f(x,y)"},"xaxis":{"title":"x"},"yaxis":{"title":"y"},"zaxis":{"title":"z"},"camera":{"eye":{"x":0.59999999999999998,"y":1.25,"z":1.3}}},"hovermode":"closest","showlegend":false,"xaxis":{"range":[-3.3500000000000001,4.3499999999999996]},"yaxis":{"range":[-2.3500000000000001,5.3499999999999996]},"sliders":[{"currentvalue":{"prefix":"iteration: ","xanchor":"right","font":{"size":16,"color":"rgba(204,204,204,1)"}},"steps":[{"method":"animate","args":[["1"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"1","value":"1"},{"method":"animate","args":[["2"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"2","value":"2"},{"method":"animate","args":[["3"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"3","value":"3"},{"method":"animate","args":[["10"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"10","value":"10"},{"method":"animate","args":[["20"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"20","value":"20"},{"method":"animate","args":[["30"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"30","value":"30"},{"method":"animate","args":[["40"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"40","value":"40"},{"method":"animate","args":[["50"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"50","value":"50"},{"method":"animate","args":[["60"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"60","value":"60"},{"method":"animate","args":[["70"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"70","value":"70"}],"visible":true,"pad":{"t":40}}],"updatemenus":[{"type":"buttons","direction":"right","showactive":false,"y":0,"x":0,"yanchor":"top","xanchor":"right","pad":{"t":60,"r":5},"buttons":[{"label":"Play","method":"animate","args":[null,{"fromcurrent":true,"mode":"immediate","transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true}}]}]}]},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false},"data":[{"colorbar":{"title":"z = f(x,y)","ticklen":2},"colorscale":[["0","rgba(68,1,84,1)"],["0.0416666666666667","rgba(70,19,97,1)"],["0.0833333333333333","rgba(72,32,111,1)"],["0.125","rgba(71,45,122,1)"],["0.166666666666667","rgba(68,58,128,1)"],["0.208333333333333","rgba(64,70,135,1)"],["0.25","rgba(60,82,138,1)"],["0.291666666666667","rgba(56,93,140,1)"],["0.333333333333333","rgba(49,104,142,1)"],["0.375","rgba(46,114,142,1)"],["0.416666666666667","rgba(42,123,142,1)"],["0.458333333333333","rgba(38,133,141,1)"],["0.5","rgba(37,144,140,1)"],["0.541666666666667","rgba(33,154,138,1)"],["0.583333333333333","rgba(39,164,133,1)"],["0.625","rgba(47,174,127,1)"],["0.666666666666667","rgba(53,183,121,1)"],["0.708333333333333","rgba(79,191,110,1)"],["0.75","rgba(98,199,98,1)"],["0.791666666666667","rgba(119,207,85,1)"],["0.833333333333333","rgba(147,214,70,1)"],["0.875","rgba(172,220,52,1)"],["0.916666666666667","rgba(199,225,42,1)"],["0.958333333333333","rgba(226,228,40,1)"],["1","rgba(253,231,37,1)"]],"showscale":true,"x":[-3,-2.7999999999999998,-2.6000000000000001,-2.3999999999999999,-2.2000000000000002,-2,-1.7999999999999998,-1.5999999999999999,-1.3999999999999999,-1.2,-1,-0.79999999999999982,-0.59999999999999964,-0.39999999999999991,-0.19999999999999973,0,0.20000000000000018,0.40000000000000036,0.60000000000000009,0.80000000000000027,1,1.2000000000000002,1.4000000000000004,1.6000000000000005,1.8000000000000007,2,2.2000000000000002,2.4000000000000004,2.6000000000000005,2.8000000000000007,3,3.2000000000000002,3.4000000000000004,3.6000000000000005,3.8000000000000007,4],"y":[-2,-1.8,-1.6000000000000001,-1.3999999999999999,-1.2,-1,-0.79999999999999982,-0.59999999999999987,-0.39999999999999991,-0.19999999999999996,0,0.20000000000000018,0.40000000000000036,0.60000000000000009,0.80000000000000027,1,1.2000000000000002,1.4000000000000004,1.6000000000000001,1.8000000000000003,2,2.2000000000000002,2.4000000000000004,2.6000000000000005,2.8000000000000007,3,3.2000000000000002,3.4000000000000004,3.6000000000000005,3.8000000000000007,4,4.2000000000000002,4.4000000000000004,4.6000000000000005,4.8000000000000007,5],"z":[[56,49.799999999999997,44,38.599999999999994,33.600000000000009,29,24.799999999999997,20.999999999999996,17.600000000000001,14.600000000000001,12,9.7999999999999972,7.9999999999999973,6.5999999999999996,5.5999999999999988,5,4.7999999999999998,5.0000000000000009,5.6000000000000005,6.6000000000000023,8,9.8000000000000025,12.000000000000004,14.600000000000009,17.600000000000009,21,24.800000000000004,29.000000000000007,33.600000000000009,38.600000000000016,44,49.800000000000011,56.000000000000014,62.600000000000023,69.600000000000023,77],[49.060000000000002,43.608800000000002,38.511200000000002,33.767199999999995,29.376800000000003,25.340000000000003,21.656799999999997,18.327199999999998,15.351199999999999,12.7288,10.460000000000001,8.5447999999999986,6.9831999999999974,5.7751999999999999,4.9207999999999981,4.4199999999999999,4.2728000000000002,4.4792000000000014,5.0392000000000001,5.9528000000000016,7.2199999999999998,8.8408000000000015,10.815200000000003,13.143200000000007,15.824800000000009,18.859999999999999,22.248800000000003,25.991200000000006,30.08720000000001,34.536800000000014,39.340000000000003,44.496800000000007,50.007200000000012,55.871200000000016,62.08880000000002,68.659999999999997],[42.640000000000008,37.883200000000002,33.436800000000005,29.300800000000002,25.475200000000008,21.960000000000004,18.755199999999999,15.860799999999999,13.276800000000001,11.003200000000001,9.0400000000000009,7.3871999999999991,6.0447999999999986,5.0128000000000004,4.2911999999999999,3.8800000000000003,3.7791999999999999,3.9888000000000012,4.5088000000000008,5.3392000000000017,6.4800000000000004,7.9312000000000014,9.6928000000000036,11.764800000000008,14.147200000000009,16.840000000000003,19.843200000000007,23.156800000000008,26.78080000000001,30.715200000000017,34.960000000000008,39.515200000000007,44.380800000000008,49.556800000000017,55.04320000000002,60.840000000000003],[36.739999999999995,32.623199999999997,28.776800000000001,25.200799999999994,21.895200000000003,18.859999999999999,16.095199999999995,13.600799999999996,11.376799999999999,9.4231999999999978,7.7399999999999993,6.3271999999999977,5.1847999999999983,4.3127999999999993,3.7111999999999985,3.3799999999999994,3.3191999999999995,3.5288000000000004,4.0087999999999999,4.7592000000000008,5.7799999999999994,7.0712000000000002,8.6328000000000014,10.464800000000006,12.567200000000005,14.94,17.583200000000001,20.496800000000004,23.680800000000005,27.135200000000008,30.859999999999999,34.855200000000004,39.120800000000003,43.656800000000011,48.463200000000008,53.539999999999992],[31.360000000000007,27.828800000000005,24.531200000000005,21.467200000000002,18.636800000000008,16.040000000000003,13.6768,11.5472,9.6512000000000011,7.9888000000000012,6.5600000000000005,5.3647999999999998,4.4031999999999991,3.6752000000000002,3.1807999999999996,2.9200000000000004,2.8928000000000007,3.0992000000000015,3.539200000000001,4.2128000000000023,5.120000000000001,6.2608000000000024,7.6352000000000038,9.243200000000007,11.084800000000008,13.160000000000002,15.468800000000005,18.011200000000009,20.787200000000013,23.796800000000012,27.040000000000006,30.516800000000011,34.227200000000011,38.171200000000013,42.348800000000018,46.760000000000012],[26.5,23.5,20.700000000000003,18.099999999999998,15.700000000000003,13.5,11.499999999999998,9.6999999999999993,8.0999999999999996,6.7000000000000002,5.5,4.4999999999999991,3.6999999999999988,3.0999999999999996,2.6999999999999993,2.5,2.5,2.7000000000000006,3.1000000000000001,3.7000000000000011,4.5,5.5000000000000009,6.700000000000002,8.1000000000000032,9.7000000000000046,11.5,13.500000000000002,15.700000000000005,18.100000000000005,20.70000000000001,23.5,26.500000000000007,29.700000000000006,33.100000000000009,36.70000000000001,40.5],[22.159999999999997,19.636799999999994,17.283200000000001,15.099199999999996,13.0848,11.239999999999998,9.5647999999999964,8.059199999999997,6.7231999999999985,5.5567999999999991,4.5599999999999987,3.7327999999999988,3.0751999999999988,2.5871999999999993,2.2687999999999993,2.1199999999999997,2.1407999999999996,2.3312000000000004,2.6911999999999998,3.220800000000001,3.9199999999999995,4.7888000000000002,5.8272000000000004,7.0352000000000023,8.4128000000000025,9.9599999999999991,11.6768,13.5632,15.619200000000003,17.844800000000003,20.239999999999998,22.8048,25.539200000000001,28.443200000000004,31.516800000000007,34.759999999999991],[18.339999999999993,16.239199999999993,14.280799999999997,12.464799999999997,10.791199999999998,9.259999999999998,7.8711999999999964,6.6247999999999969,5.5207999999999986,4.5591999999999988,3.7399999999999989,3.0631999999999984,2.5287999999999986,2.1367999999999991,1.8871999999999991,1.7799999999999994,1.8151999999999995,1.9928000000000001,2.3127999999999997,2.7752000000000003,3.379999999999999,4.1271999999999993,5.0167999999999999,6.0488000000000017,7.2232000000000012,8.5399999999999974,9.9991999999999983,11.600799999999998,13.344799999999999,15.231200000000003,17.259999999999994,19.431199999999997,21.744799999999998,24.200800000000001,26.799200000000003,29.539999999999988],[15.039999999999999,13.3072,11.6928,10.1968,8.8192000000000004,7.5599999999999987,6.4191999999999991,5.3967999999999989,4.492799999999999,3.7071999999999998,3.04,2.4911999999999992,2.0607999999999995,1.7487999999999999,1.5551999999999995,1.48,1.5232000000000001,1.6848000000000005,1.9648000000000001,2.3632000000000004,2.8799999999999999,3.5152000000000001,4.2688000000000015,5.1408000000000023,6.1312000000000024,7.2399999999999993,8.4672000000000018,9.8128000000000029,11.276800000000003,12.859200000000005,14.559999999999999,16.379200000000001,18.316800000000001,20.372800000000005,22.547200000000007,24.84],[12.260000000000002,10.840800000000002,9.5192000000000014,8.2952000000000012,7.1688000000000018,6.1400000000000006,5.2088000000000001,4.3751999999999995,3.6392000000000002,3.0008000000000004,2.4600000000000004,2.0167999999999999,1.6711999999999998,1.4232000000000002,1.2727999999999999,1.2200000000000002,1.2648000000000001,1.4072000000000007,1.6472000000000004,1.984800000000001,2.4200000000000004,2.9528000000000008,3.5832000000000015,4.3112000000000021,5.1368000000000036,6.0600000000000005,7.0808000000000018,8.1992000000000029,9.4152000000000058,10.728800000000007,12.140000000000001,13.648800000000003,15.255200000000006,16.959200000000006,18.760800000000007,20.660000000000004],[10,8.8399999999999999,7.7600000000000007,6.7599999999999998,5.8400000000000007,5,4.2399999999999993,3.5599999999999996,2.96,2.4399999999999999,2,1.6399999999999997,1.3599999999999997,1.1599999999999999,1.0399999999999998,1,1.04,1.1600000000000004,1.3600000000000001,1.6400000000000006,2,2.4400000000000004,2.9600000000000009,3.5600000000000018,4.240000000000002,5,5.8400000000000007,6.7600000000000016,7.7600000000000025,8.8400000000000034,10,11.240000000000002,12.560000000000002,13.960000000000004,15.440000000000005,17],[8.259999999999998,7.3047999999999984,6.4151999999999996,5.5911999999999988,4.8327999999999998,4.1399999999999988,3.512799999999999,2.9511999999999992,2.4551999999999996,2.0247999999999995,1.6599999999999997,1.3607999999999993,1.1271999999999995,0.95919999999999983,0.85679999999999967,0.81999999999999984,0.84879999999999978,0.94320000000000015,1.1032,1.3288000000000002,1.6199999999999997,1.9767999999999999,2.3992,2.8872000000000009,3.4408000000000012,4.0599999999999987,4.7447999999999997,5.4951999999999996,6.3112000000000004,7.192800000000001,8.1399999999999988,9.1528000000000009,10.231200000000001,11.375200000000001,12.584800000000001,13.859999999999998],[7.0399999999999983,6.2351999999999981,5.484799999999999,4.7887999999999984,4.1471999999999998,3.5599999999999992,3.0271999999999988,2.5487999999999991,2.1247999999999996,1.7551999999999996,1.4399999999999997,1.1791999999999994,0.97279999999999955,0.82079999999999975,0.72319999999999962,0.67999999999999972,0.6911999999999997,0.75679999999999981,0.87679999999999958,1.0511999999999999,1.2799999999999994,1.5631999999999995,1.9007999999999996,2.2928000000000002,2.7392000000000003,3.2399999999999984,3.795199999999999,4.4047999999999989,5.0687999999999995,5.7871999999999995,6.5599999999999969,7.3871999999999973,8.2687999999999988,9.2047999999999988,10.1952,11.239999999999997],[6.3399999999999999,5.6311999999999998,4.9687999999999999,4.3527999999999993,3.7832000000000008,3.2599999999999998,2.7831999999999995,2.3527999999999993,1.9687999999999999,1.6312,1.3399999999999999,1.0951999999999997,0.89679999999999971,0.74479999999999991,0.63919999999999977,0.57999999999999996,0.56719999999999993,0.60080000000000011,0.68079999999999996,0.80720000000000014,0.97999999999999987,1.1992,1.4648000000000001,1.7768000000000006,2.1352000000000011,2.5399999999999996,2.9912000000000001,3.4888000000000003,4.0328000000000008,4.6232000000000006,5.2599999999999998,5.9432,6.6728000000000005,7.4488000000000021,8.2712000000000021,9.1399999999999988],[6.1600000000000001,5.4928000000000008,4.8672000000000004,4.2831999999999999,3.740800000000001,3.2400000000000002,2.7807999999999997,2.3632,1.9872000000000001,1.6528,1.3600000000000001,1.1088,0.89919999999999978,0.73119999999999985,0.60479999999999978,0.51999999999999991,0.47679999999999989,0.4751999999999999,0.51519999999999977,0.59679999999999989,0.71999999999999964,0.8847999999999997,1.0911999999999999,1.3392000000000002,1.6288000000000002,1.9599999999999993,2.3327999999999998,2.7471999999999999,3.2032000000000003,3.7008000000000001,4.2399999999999984,4.8207999999999993,5.4431999999999992,6.1072000000000006,6.8128000000000002,7.5599999999999978],[6.5,5.8200000000000003,5.1800000000000006,4.5800000000000001,4.0200000000000005,3.5,3.0199999999999996,2.5799999999999992,2.1799999999999997,1.8200000000000003,1.5,1.2199999999999998,0.97999999999999954,0.77999999999999992,0.61999999999999977,0.5,0.41999999999999993,0.38,0.38,0.4200000000000001,0.5,0.62000000000000011,0.78000000000000025,0.98000000000000065,1.2200000000000009,1.5,1.8200000000000003,2.1800000000000006,2.580000000000001,3.0200000000000014,3.5,4.0200000000000005,4.580000000000001,5.1800000000000015,5.8200000000000021,6.5],[7.3600000000000012,6.6128,5.9072000000000013,5.2431999999999999,4.6208000000000009,4.0400000000000009,3.5007999999999999,3.0031999999999996,2.5472000000000001,2.1328000000000005,1.7600000000000002,1.4287999999999998,1.1391999999999995,0.89119999999999999,0.68479999999999974,0.52000000000000002,0.39679999999999982,0.31519999999999981,0.27519999999999989,0.27679999999999993,0.31999999999999984,0.40479999999999988,0.53120000000000001,0.69920000000000027,0.9088000000000005,1.1599999999999997,1.4527999999999999,1.7872000000000003,2.1632000000000007,2.5808000000000009,3.0399999999999996,3.5407999999999999,4.0832000000000006,4.6672000000000011,5.2928000000000015,5.96],[8.7400000000000038,7.8712000000000026,7.0488000000000035,6.2728000000000019,5.5432000000000032,4.8600000000000021,4.2232000000000012,3.6328000000000005,3.0888000000000009,2.5912000000000015,2.140000000000001,1.7352000000000003,1.3767999999999998,1.0648000000000002,0.79919999999999991,0.58000000000000018,0.4071999999999999,0.28079999999999983,0.20079999999999987,0.16719999999999985,0.1799999999999998,0.23919999999999983,0.3448,0.49680000000000024,0.69520000000000048,0.93999999999999972,1.2312000000000001,1.5688000000000004,1.952800000000001,2.3832000000000013,2.8600000000000003,3.3832000000000004,3.9528000000000012,4.5688000000000022,5.231200000000003,5.9400000000000013],[10.640000000000002,9.5952000000000019,8.6048000000000009,7.6688000000000001,6.787200000000003,5.9600000000000017,5.1872000000000007,4.4687999999999999,3.8048000000000006,3.1952000000000012,2.6400000000000006,2.1391999999999998,1.6927999999999994,1.3008000000000002,0.96319999999999972,0.68000000000000016,0.45119999999999982,0.27679999999999977,0.15679999999999994,0.091199999999999934,0.07999999999999996,0.12320000000000002,0.22080000000000016,0.37280000000000046,0.57920000000000083,0.84000000000000008,1.1552000000000002,1.5248000000000008,1.9488000000000014,2.4272000000000022,2.9600000000000004,3.547200000000001,4.1888000000000014,4.884800000000002,5.6352000000000029,6.4400000000000013],[13.060000000000004,11.784800000000004,10.575200000000004,9.4312000000000022,8.3528000000000056,7.3400000000000025,6.3928000000000011,5.5111999999999997,4.6952000000000016,3.9448000000000021,3.2600000000000011,2.6408000000000005,2.0871999999999997,1.5992000000000004,1.1767999999999998,0.82000000000000028,0.52879999999999994,0.30319999999999975,0.14319999999999997,0.048799999999999892,0.019999999999999948,0.05680000000000001,0.1592000000000002,0.32720000000000055,0.56080000000000096,0.8600000000000001,1.2248000000000006,1.6552000000000013,2.151200000000002,2.7128000000000023,3.3400000000000007,4.0328000000000017,4.7912000000000026,5.6152000000000033,6.5048000000000057,7.4600000000000026],[16,14.44,12.960000000000001,11.559999999999999,10.240000000000002,9,7.839999999999999,6.759999999999998,5.7599999999999998,4.8400000000000007,4,3.2399999999999993,2.5599999999999987,1.9599999999999997,1.4399999999999993,1,0.63999999999999968,0.3599999999999996,0.15999999999999992,0.039999999999999897,0,0.04000000000000007,0.16000000000000028,0.36000000000000065,0.64000000000000112,1,1.4400000000000004,1.9600000000000011,2.5600000000000018,3.2400000000000024,4,4.8400000000000007,5.7600000000000016,6.7600000000000025,7.8400000000000043,9],[19.460000000000004,17.5608,15.759200000000003,14.055200000000001,12.448800000000006,10.940000000000003,9.5288000000000004,8.2151999999999994,6.999200000000001,5.8808000000000016,4.8600000000000012,3.9367999999999999,3.1111999999999989,2.3832,1.7527999999999995,1.2200000000000002,0.78479999999999972,0.4471999999999996,0.20719999999999997,0.064799999999999913,0.020000000000000035,0.072800000000000142,0.22320000000000045,0.47120000000000095,0.81680000000000164,1.2600000000000002,1.8008000000000008,2.4392000000000018,3.1752000000000029,4.0088000000000035,4.9400000000000013,5.9688000000000017,7.0952000000000037,8.3192000000000039,9.6408000000000076,11.060000000000002],[23.440000000000008,21.147200000000005,18.972800000000007,16.916800000000006,14.979200000000008,13.160000000000005,11.459200000000003,9.8767999999999994,8.4128000000000007,7.0672000000000033,5.8400000000000016,4.7312000000000003,3.7407999999999992,2.8688000000000007,2.1151999999999993,1.4800000000000004,0.96319999999999983,0.56479999999999964,0.28480000000000005,0.1232,0.08000000000000014,0.15520000000000031,0.34880000000000072,0.66080000000000139,1.0912000000000022,1.6400000000000008,2.3072000000000017,3.0928000000000027,3.9968000000000039,5.0192000000000059,6.1600000000000019,7.4192000000000045,8.7968000000000064,10.292800000000007,11.90720000000001,13.640000000000006],[27.940000000000012,25.199200000000008,22.600800000000014,20.144800000000004,17.83120000000001,15.660000000000007,13.631200000000003,11.744800000000001,10.000800000000003,8.399200000000004,6.9400000000000031,5.6232000000000015,4.4487999999999994,3.4168000000000007,2.5272000000000001,1.7800000000000007,1.1752,0.71279999999999977,0.3928000000000002,0.21520000000000017,0.18000000000000033,0.28720000000000057,0.53680000000000117,0.92880000000000207,1.4632000000000032,2.1400000000000015,2.9592000000000027,3.9208000000000043,5.0248000000000062,6.2712000000000083,7.6600000000000037,9.1912000000000056,10.864800000000008,12.68080000000001,14.639200000000015,16.740000000000009],[32.960000000000022,29.716800000000017,26.643200000000018,23.739200000000011,21.004800000000014,18.440000000000012,16.044800000000006,13.819200000000004,11.763200000000007,9.8768000000000082,8.1600000000000055,6.6128000000000027,5.2351999999999999,4.0272000000000014,2.9888000000000003,2.1200000000000014,1.4208000000000003,0.89119999999999999,0.53120000000000034,0.34080000000000044,0.32000000000000056,0.46880000000000094,0.78720000000000168,1.275200000000003,1.9328000000000043,2.7600000000000025,3.7568000000000041,4.9232000000000067,6.2592000000000088,7.7648000000000117,9.4400000000000066,11.284800000000011,13.299200000000013,15.483200000000016,17.836800000000022,20.360000000000017],[38.5,34.700000000000003,31.100000000000005,27.699999999999999,24.500000000000007,21.5,18.699999999999996,16.099999999999994,13.699999999999999,11.500000000000002,9.5,7.6999999999999984,6.099999999999997,4.6999999999999993,3.4999999999999982,2.5,1.6999999999999993,1.0999999999999992,0.69999999999999984,0.49999999999999989,0.5,0.70000000000000029,1.100000000000001,1.700000000000002,2.5000000000000031,3.5,4.7000000000000011,6.1000000000000032,7.7000000000000046,9.5000000000000071,11.5,13.700000000000003,16.100000000000005,18.700000000000006,21.500000000000014,24.5],[44.560000000000009,40.148800000000008,35.971200000000003,32.027200000000001,28.316800000000011,24.840000000000007,21.596800000000002,18.587199999999999,15.811200000000003,13.268800000000006,10.960000000000003,8.8848000000000003,7.0431999999999979,5.4352000000000009,4.0607999999999986,2.9200000000000008,2.0127999999999995,1.3391999999999995,0.8992,0.69280000000000008,0.7200000000000002,0.98080000000000056,1.4752000000000014,2.2032000000000029,3.1648000000000041,4.3600000000000012,5.7888000000000019,7.4512000000000045,9.3472000000000079,11.476800000000011,13.840000000000003,16.436800000000005,19.26720000000001,22.33120000000001,25.628800000000016,29.160000000000007],[51.140000000000015,46.063200000000009,41.256800000000013,36.720800000000004,32.455200000000012,28.460000000000008,24.735200000000003,21.280799999999999,18.096800000000005,15.183200000000005,12.540000000000003,10.167200000000001,8.0647999999999982,6.232800000000001,4.6711999999999989,3.3800000000000008,2.3591999999999995,1.6087999999999996,1.1288000000000002,0.91920000000000046,0.98000000000000054,1.311200000000001,1.9128000000000021,2.7848000000000042,3.9272000000000058,5.3400000000000016,7.0232000000000037,8.9768000000000061,11.20080000000001,13.695200000000014,16.460000000000004,19.495200000000008,22.80080000000001,26.376800000000017,30.223200000000023,34.340000000000011],[58.240000000000023,52.443200000000019,46.956800000000015,41.780800000000013,36.91520000000002,32.360000000000014,28.115200000000009,24.180800000000001,20.556800000000006,17.243200000000009,14.240000000000006,11.547200000000002,9.1647999999999996,7.0928000000000022,5.3311999999999999,3.8800000000000017,2.7392000000000003,1.9087999999999998,1.3888000000000007,1.1792000000000009,1.2800000000000009,1.6912000000000016,2.4128000000000029,3.4448000000000052,4.7872000000000074,6.4400000000000031,8.4032000000000053,10.676800000000011,13.260800000000014,16.155200000000018,19.36000000000001,22.875200000000014,26.700800000000019,30.836800000000025,35.283200000000036,40.04000000000002],[65.860000000000028,59.288800000000016,53.071200000000019,47.207200000000014,41.696800000000025,36.540000000000013,31.736800000000009,27.287200000000002,23.191200000000006,19.448800000000009,16.060000000000006,13.024800000000003,10.3432,8.0152000000000019,6.0407999999999999,4.4200000000000017,3.1528,2.2391999999999999,1.6792000000000007,1.472800000000001,1.6200000000000012,2.1208000000000018,2.9752000000000036,4.1832000000000065,5.7448000000000086,7.6600000000000046,9.9288000000000061,12.551200000000012,15.527200000000015,18.856800000000021,22.540000000000013,26.576800000000016,30.96720000000002,35.711200000000034,40.808800000000033,46.260000000000019],[74,66.599999999999994,59.600000000000001,52.999999999999993,46.800000000000011,41,35.599999999999994,30.599999999999991,26,21.800000000000004,18,14.599999999999998,11.599999999999994,8.9999999999999982,6.7999999999999972,5,3.5999999999999988,2.5999999999999988,1.9999999999999998,1.8000000000000003,2,2.6000000000000005,3.6000000000000019,5.0000000000000044,6.800000000000006,9,11.600000000000001,14.600000000000005,18.000000000000011,21.800000000000011,26,30.600000000000005,35.600000000000009,41.000000000000014,46.800000000000026,53],[82.659999999999997,74.376800000000003,66.543200000000013,59.159199999999998,52.224800000000009,45.739999999999995,39.704799999999999,34.119199999999992,28.9832,24.296800000000005,20.060000000000002,16.272799999999997,12.935199999999995,10.0472,7.6087999999999969,5.6200000000000001,4.0807999999999991,2.9911999999999992,2.3512,2.1608000000000005,2.4200000000000004,3.1288000000000009,4.287200000000003,5.8952000000000053,7.9528000000000088,10.460000000000001,13.416800000000004,16.823200000000007,20.679200000000016,24.984800000000018,29.740000000000002,34.944800000000008,40.59920000000001,46.703200000000024,53.256800000000027,60.260000000000005],[91.840000000000018,82.619200000000006,73.900800000000018,65.68480000000001,57.971200000000024,50.760000000000012,44.051200000000001,37.844799999999999,32.140800000000006,26.939200000000007,22.240000000000002,18.043199999999999,14.348799999999997,11.1568,8.4671999999999983,6.2800000000000011,4.5951999999999993,3.4127999999999994,2.7328000000000001,2.555200000000001,2.8800000000000008,3.707200000000002,5.0368000000000039,6.8688000000000073,9.2032000000000096,12.040000000000003,15.379200000000006,19.220800000000011,23.56480000000002,28.411200000000022,33.760000000000005,39.611200000000011,45.964800000000025,52.820800000000027,60.179200000000037,68.04000000000002],[101.54000000000002,91.327200000000005,81.672800000000024,72.576800000000006,64.039200000000022,56.060000000000009,48.639200000000002,41.776800000000001,35.472800000000007,29.727200000000011,24.540000000000006,19.911199999999997,15.840799999999996,12.328800000000001,9.3751999999999978,6.9800000000000013,5.1431999999999993,3.8647999999999998,3.1448000000000005,2.9832000000000014,3.3800000000000012,4.3352000000000031,5.8488000000000051,7.9208000000000087,10.551200000000012,13.740000000000004,17.487200000000009,21.792800000000014,26.656800000000018,32.079200000000029,38.060000000000009,44.599200000000017,51.696800000000025,59.35280000000003,67.567200000000042,76.340000000000018],[111.76000000000003,100.50080000000003,89.85920000000003,79.835200000000015,70.428800000000038,61.640000000000022,53.468800000000009,45.915199999999999,38.979200000000006,32.660800000000016,26.960000000000008,21.876800000000003,17.411199999999997,13.563200000000002,10.332799999999999,7.7200000000000024,5.7248000000000001,4.3472,3.5872000000000015,3.4448000000000025,3.9200000000000021,5.0128000000000039,6.7232000000000065,9.0512000000000121,11.996800000000016,15.560000000000008,19.740800000000014,24.539200000000019,29.955200000000026,35.98880000000004,42.640000000000015,49.908800000000028,57.795200000000037,66.299200000000042,75.420800000000057,85.160000000000039],[122.5,110.14,98.460000000000008,87.45999999999998,77.140000000000015,67.5,58.539999999999992,50.259999999999984,42.659999999999997,35.740000000000002,29.5,23.939999999999998,19.059999999999992,14.859999999999998,11.339999999999995,8.5,6.3399999999999981,4.8599999999999977,4.0599999999999996,3.9400000000000008,4.5,5.740000000000002,7.6600000000000037,10.260000000000007,13.540000000000012,17.5,22.140000000000004,27.460000000000012,33.460000000000015,40.140000000000022,47.5,55.540000000000006,64.260000000000019,73.660000000000025,83.740000000000038,94.5]],"showlegend":false,"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"frame":null},{"x":[-2,-0.59999999999999987,-0.26426249999999996,-0.041499230184818769,0.12362438742854515,0.25294706198441524,0.35759050325897518,0.44408362624962455,0.516634206852428,0.57814070100131087,0.63070135151152384,0.67589406574763289,0.71494113784348157,0.74881167724783815,0.77828829775719699,0.80401228564807825,0.82651526545406462,0.84624208449199567,0.86356779970433972,0.87881058553288038,0.8922417435434784,0.90409360103332836,0.91456583681346348,0.92383061096626884,0.93203676842795791,0.93931331383889494,0.9457723050491027,0.95151127734773899,0.9566152850632631,0.96115862852726042,0.96520632045542609,0.96881533520392438,0.97203567617966335,0.97491129027935375,0.97748085315660271,0.97977844504973244,0.98183413361444571,0.98367447752476067,0.9853229624055122,0.98680037884348315,0.98812515071742202,0.98931362083173435,0.99038029978868358,0.99133808315298588,0.99219844122139156,0.99297158508444427,0.99366661213873175,0.99429163375971463,0.99485388746457115,0.99535983557061836,0.99581525207879362,0.99622529927593984,0.99659459534800265,0.99692727412349691,0.99722703791836742,0.99749720432597067,0.9977407476862491,0.99796033587364297,0.99815836296168126,0.99833697825164425,0.99849811209160744,0.99864349885922432,0.99877469743562752,0.99889310945786114,0.99899999560246289,0.99909649012248714,0.9991836138337904,0.99926228572327058,0.99933333333151131,0.99939750204455491,0.99945546341397706,0.99950782261078175,0.99955512510663402],"y":[4,2.8499999999999996,2.6386999999999996,2.5161776234656661,2.4350172135714154,2.3772211608366516,2.3340197751097698,2.3005208316540537,2.2737521687013613,2.2518005505578329,2.2333828921846104,2.2176126169702646,2.2038632781860188,2.1916845564687124,2.180748273731818,2.1708125126516506,2.16169712996293,2.153266695445172,2.1454184115214474,2.1380734523103317,2.1311706946567419,2.1246621474319132,2.1185096007200723,2.1126821592439549,2.1071544212203244,2.1019051308360788,2.0969161796478613,2.0921718657779942,2.0876583439568774,2.0833632170242118,2.0792752323453079,2.0753840560423784,2.0716801049134208,2.0681544210763652,2.0647985782117284,2.0616046111305666,2.0585649625202325,2.0556724423052657,2.0529201962427654,2.0503016812531394,2.0478106456442595,2.0454411128764818,2.0431873678799723,2.0410439452059643,2.0390056184936105,2.0370673908817087,2.0352244861030688,2.0334723400785801,2.0318065928856437,2.030223081017104,2.0287178298763782,2.0272870464752475,2.025927112315121,2.0246345764422156,2.0234061486733683,2.0222386929930352,2.0211292211241618,2.0200748862765776,2.0190729770767235,2.0181209116821677,2.0172162320837068,2.016356598596992,2.0155397845447229,2.0147636711295154,2.0140262424966755,2.0133255809852924,2.0126598625653211,2.0120273524576886,2.0114264009338898,2.0108554392910851,2.0103129759983123,2.0097975930091287,2.0093079422357629],"z":[41,5.4440499999999981,2.8913462480962488,1.7836067841736578,1.1865110443893503,0.82631417992814737,0.59350636116364341,0.43592535501466939,0.32571558988524696,0.24674690958568113,0.18910116967162696,0.14639447387475354,0.11436931391869382,0.090105962054926547,0.071557233749252408,0.057261953826898006,0.046162019249196801,0.037482029028616262,0.030648058924578312,0.025231683686573473,0.020910715121260297,0.017441257760386939,0.014637574429919996,0.012357428701896551,0.010491320756246984,0.0089545229850728839,0.0076811485906396583,0.006619708727239524,0.0057297673655053241,0.004979410725466803,0.0043433244949027863,0.0038013267817681314,0.0033372443231640962,0.0029380483058181021,0.0025931872991705783,0.0022940704010762244,0.0020336652621170269,0.0018061842697103738,0.0016068386171883703,0.0014316448214855347,0.0012772718989798406,0.0011409201655421895,0.0010202247175771142,0.00091317824145082575,0.00081806901242300874,0.00073343087300964328,0.00065800269350525429,0.00059069536595283522,0.00053056480618767019,0.00047678976618587042,0.00042865351317217698,0.00038552862976925931,0.00034686434384504576,0.00031217591752049105,0.00028103571960904982,0.00025306568038163321,0.00022793088645958702,0.00020533412028320852,0.0001850111856517496,0.0001667268903503864,0.000150271580477211,0.00013545814000887423,0.00012211938437522025,0.00011010578911381,9.9283504645466126e-05,8.9532616322429449e-05,8.0745615523387129e-05,7.2826052998265279e-05,6.5687350132682338e-05,5.9251747492485029e-05,5.344937306969237e-05,4.8217415200422887e-05,4.3499387256816659e-05],"type":"scatter3d","mode":"lines","showlegend":false,"line":{"color":"#07FAFA","width":5},"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"frame":null},{"x":[-2],"y":[4],"z":[41],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"1","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.20000000000000001,"selected":{"opacity":1},"debounce":0},"frames":[{"name":"1","data":[{"x":[-2],"y":[4],"z":[41],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"1","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"2","data":[{"x":[-0.59999999999999987],"y":[2.8499999999999996],"z":[5.4440499999999981],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"2","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"3","data":[{"x":[-0.26426249999999996],"y":[2.6386999999999996],"z":[2.8913462480962488],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"3","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"10","data":[{"x":[0.57814070100131087],"y":[2.2518005505578329],"z":[0.24674690958568113],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"10","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"20","data":[{"x":[0.87881058553288038],"y":[2.1380734523103317],"z":[0.025231683686573473],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"20","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"30","data":[{"x":[0.96115862852726042],"y":[2.0833632170242118],"z":[0.004979410725466803],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"30","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"40","data":[{"x":[0.98680037884348315],"y":[2.0503016812531394],"z":[0.0014316448214855347],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"40","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"50","data":[{"x":[0.99535983557061836],"y":[2.030223081017104],"z":[0.00047678976618587042],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"50","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"60","data":[{"x":[0.99833697825164425],"y":[2.0181209116821677],"z":[0.0001667268903503864],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"60","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"70","data":[{"x":[0.99939750204455491],"y":[2.0108554392910851],"z":[5.9251747492485029e-05],"showlegend":false,"type":"scatter3d","mode":"markers","frame":"70","marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]}],"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; `Tip ~ Total_bill` using GD <h0br>

.pull-left[
<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-898f534ac2b0f7327c06" style="width:400px;height:400px;"></div>
<script type="application/json" data-for="htmlwidget-898f534ac2b0f7327c06">{"x":{"visdat":{"ee8609d3689":["function () ","plotlyVisDat"]},"cur_data":"ee8609d3689","attrs":{"ee8609d3689":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"inherit":true},"ee8609d3689.1":{"x":{},"y":{},"z":{},"type":"scatter3d","mode":"lines","showlegend":false,"line":{"color":"#07FAFA","width":5,"type":"dashed"},"inherit":false},"ee8609d3689.2":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"scatter3d","mode":"markers","frame":{},"showlegend":false,"marker":{"color":"#70EF44"},"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"camera":{"eye":{"x":0,"y":-1.25,"z":0.69999999999999996}},"xaxis":{"title":"a"},"yaxis":{"title":"b"},"zaxis":{"title":"Loss"}},"hovermode":"closest","showlegend":false,"xaxis":{"range":[-0.22500000000000001,0.32500000000000001]},"yaxis":{"range":[-8.9000000000000004,10.9]},"sliders":[{"currentvalue":{"prefix":"iteration: ","xanchor":"right","font":{"size":16,"color":"rgba(204,204,204,1)"}},"steps":[{"method":"animate","args":[["1"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"1","value":"1"},{"method":"animate","args":[["2"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"2","value":"2"},{"method":"animate","args":[["3"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"3","value":"3"},{"method":"animate","args":[["4"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"4","value":"4"},{"method":"animate","args":[["5"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"5","value":"5"},{"method":"animate","args":[["6"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"6","value":"6"},{"method":"animate","args":[["7"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"7","value":"7"},{"method":"animate","args":[["8"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"8","value":"8"},{"method":"animate","args":[["9"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"9","value":"9"},{"method":"animate","args":[["10"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"10","value":"10"},{"method":"animate","args":[["11"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"11","value":"11"},{"method":"animate","args":[["12"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"12","value":"12"},{"method":"animate","args":[["13"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"13","value":"13"},{"method":"animate","args":[["14"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"14","value":"14"},{"method":"animate","args":[["15"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"15","value":"15"}],"visible":true,"pad":{"t":40}}],"updatemenus":[{"type":"buttons","direction":"right","showactive":false,"y":0,"x":0,"yanchor":"top","xanchor":"right","pad":{"t":60,"r":5},"buttons":[{"label":"Play","method":"animate","args":[null,{"fromcurrent":true,"mode":"immediate","transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true}}]}]}],"legend":{"yanchor":"top","y":0.5}},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false},"data":[{"colorbar":{"title":"loss = f(x,y)","ticklen":2,"len":0.5,"lenmode":"fraction","y":1,"yanchor":"top"},"colorscale":[["0","rgba(68,1,84,1)"],["0.0416666666666667","rgba(70,19,97,1)"],["0.0833333333333333","rgba(72,32,111,1)"],["0.125","rgba(71,45,122,1)"],["0.166666666666667","rgba(68,58,128,1)"],["0.208333333333333","rgba(64,70,135,1)"],["0.25","rgba(60,82,138,1)"],["0.291666666666667","rgba(56,93,140,1)"],["0.333333333333333","rgba(49,104,142,1)"],["0.375","rgba(46,114,142,1)"],["0.416666666666667","rgba(42,123,142,1)"],["0.458333333333333","rgba(38,133,141,1)"],["0.5","rgba(37,144,140,1)"],["0.541666666666667","rgba(33,154,138,1)"],["0.583333333333333","rgba(39,164,133,1)"],["0.625","rgba(47,174,127,1)"],["0.666666666666667","rgba(53,183,121,1)"],["0.708333333333333","rgba(79,191,110,1)"],["0.75","rgba(98,199,98,1)"],["0.791666666666667","rgba(119,207,85,1)"],["0.833333333333333","rgba(147,214,70,1)"],["0.875","rgba(172,220,52,1)"],["0.916666666666667","rgba(199,225,42,1)"],["0.958333333333333","rgba(226,228,40,1)"],["1","rgba(253,231,37,1)"]],"showscale":true,"x":[-0.20000000000000001,-0.1736842105263158,-0.14736842105263159,-0.12105263157894738,-0.094736842105263175,-0.068421052631578966,-0.042105263157894757,-0.015789473684210548,0.010526315789473661,0.03684210526315787,0.06315789473684208,0.089473684210526316,0.1157894736842105,0.14210526315789468,0.16842105263157892,0.19473684210526315,0.22105263157894733,0.24736842105263152,0.27368421052631575,0.29999999999999999],"y":[-8,-7.0526315789473681,-6.1052631578947372,-5.1578947368421053,-4.2105263157894743,-3.2631578947368425,-2.3157894736842106,-1.3684210526315796,-0.42105263157894779,0.52631578947368318,1.473684210526315,2.4210526315789469,3.3684210526315788,4.3157894736842106,5.2631578947368407,6.2105263157894726,7.1578947368421044,8.1052631578947363,9.0526315789473664,10],"z":[[232.04548890163935,215.53004656858226,199.66614392661552,184.45378097573908,169.89295771595297,155.98367414725718,142.72593026965171,130.11972608313656,118.16506158771173,106.86193678337725,96.210351670133065,86.210306247979204,76.861800516915679,68.164834476942488,60.11940812805959,52.725521470267019,45.98317450356479,39.892367227952874,34.45309964343128,29.665371750000002],[204.60632110726124,189.07743547008536,174.20008952399982,159.97428326900459,146.40001670509969,133.47728983228509,121.20610265056084,109.58645515992689,98.618347360383268,88.301779251929986,78.63675083456701,69.623262108294355,61.261313073112035,53.550903729020035,46.492034076018349,40.084704114106984,34.32891384328596,29.22466326355525,24.771952374914857,20.970781177364788],[178.96216716329869,164.41983822200402,150.52904897179965,137.28979941268562,124.70208954466193,112.76591936772854,101.48128888188549,90.848198087132758,80.866646983470332,71.536635570898241,62.858163849416478,54.831231819025028,47.455839479723913,40.731986831513119,34.659673874392638,29.238900608362478,24.469667033422652,20.351973149573148,16.885818956813957,14.071204455145091],[155.11302706975161,141.55725482433814,128.65302227001499,116.40032940678216,104.79917623463967,93.849562753587492,83.551488963625644,73.904954864754103,64.909960456972897,56.566505740282011,48.874590714681446,41.834215380171202,35.445379736751292,29.708083784421703,24.622327523182424,20.188110953033469,16.405434073974849,13.274296886006546,10.794699389128562,8.9666415833409019],[133.05890082662006,120.4896852770878,108.57200941864586,97.30587325129423,86.691276775032946,76.728219989861969,67.416702895781313,58.756725492790991,50.748287780890983,43.391389760081303,36.686031430361943,30.632212791732901,25.229933844194189,20.479194587745802,16.379995022387732,12.93233514811998,10.136214964942562,7.9916344728554636,6.4985936718586848,5.6570925619522292],[112.79978843390401,101.21712958025296,90.286010417692211,80.006430946221798,70.378391165841705,61.401891076551941,53.076930678352497,45.403509971243366,38.381628955224571,32.011287630296088,26.292485996457934,21.225224053710097,16.809501802052594,13.04531924148541,9.932676372008542,7.4715731936219996,5.662009706325783,4.503985910119888,3.997501805004315,4.1425573909790652],[94.335689891603479,83.739587733833616,73.795025267154088,64.50200249156488,55.860519407065986,47.870576013657427,40.532172311339181,33.845308300111263,27.809983979973669,22.426199350926396,17.693954412969443,13.613249166102811,10.18408361032651,7.4064577456405312,5.280371572044869,3.8058250895395309,2.9828182981245184,2.8113511977998269,3.2914237885654591,4.4230360704214151],[77.666605199718461,68.057059737829817,59.09905396703148,50.792587887323478,43.137661498705796,36.134274801178435,29.782427794741395,24.082120479394678,19.033352855138286,14.636124921972216,10.890436679896471,7.7962881289110433,5.3536792690159443,3.5626101002111676,2.4230806224967103,1.9350908358725769,2.0986407403387668,2.913730335895278,4.3803596225421151,6.4985286002792755],[62.792534358248957,54.169545592241505,46.19809651732438,38.878187133497583,32.2098174407611,26.192987439114944,20.827697128559109,16.113946509093598,12.051735580718411,8.6410643434335466,5.8819327972390036,3.7743409421347818,2.3182887781208867,1.5137763051973132,1.3608035233640614,1.8593704326211338,3.0094770329685274,4.8111233244062417,7.2643093069342841,10.36903498055265],[49.713477367194969,42.077045297068722,35.092152918032802,28.758800230087203,23.076987233231929,18.046713927466975,13.667980312792345,9.9407863892080375,6.8651321567140524,4.4410176153103906,2.6684427649970521,1.5474076057740354,1.077912137641343,1.2599563605989719,2.0935402746469247,3.5786638797852017,5.7153271760137976,8.5035301633327158,11.943272841741962,16.034555211241532],[38.429434226556481,31.779558852311439,25.781223169156725,20.434427177092328,15.739170876118258,11.69545426623451,8.3032773474410853,5.5626401197379822,3.473542583125202,2.0359847376027451,1.2499665831706106,1.1154881198287994,1.6325493475773105,2.8011502664161427,4.6212908763453013,7.0929711773647837,10.216191169474584,13.990950852674704,18.417250226965155,23.495089292345934],[28.940404936333504,23.277086257969671,18.265307270696159,13.905067974512971,10.196368369420103,7.1392084554175597,4.7335882325053387,2.9795077006834405,1.8769668599518652,1.4259657103106125,1.6265042517596828,2.4785824842990771,3.9822004079287914,6.1373580226488276,8.944055328459191,12.40229232535988,16.512069013350882,21.273385392432207,26.686241462603867,32.750637223865844],[21.246389496526046,16.569627514043415,12.544405222651109,9.1707226223491229,6.4485797131374625,4.3779764950161226,2.9589129679851061,2.1913891320444123,2.0754049871940414,2.6109605334339938,3.7980557707642681,5.636690699184868,8.1268653186957867,11.268579629297026,15.061833630988595,19.506627323770488,24.602960707642694,30.350833782605225,36.75024654865809,43.801199005801273],[15.347387907134101,11.657182620532675,8.6185170250215712,6.231391120600791,4.4958049072703341,3.4117583850301991,2.979251553880387,3.1982844138208981,4.0688569648517321,5.590969206972888,7.7646211401843672,10.589812764486172,14.066544079878295,18.194815086360737,22.974625783933512,28.405976172596613,34.488866252350022,41.223296023193754,48.609265485127821,56.646774638152216],[11.243400168157674,8.5397515774374497,6.4876426778075498,5.0870734692679722,4.3380439518187179,4.2405541254597861,4.7946039901911766,6.0001935460128903,7.8573227929249265,10.365991730927286,13.526200360019967,17.337948680202977,21.801236691476301,26.916064393839946,32.682431787293922,39.100338871838225,46.169785647472835,53.890772114197773,62.263298272013046,71.287364120918639],[8.9344262795967513,7.2173343847577316,6.151782181009037,5.7377696683506629,5.975296846782614,6.8643637163048856,8.4049702769174814,10.597116528620399,13.44080247141364,16.936028105297204,21.082793430271092,25.881098446335304,31.33094315348983,37.43232755173468,44.185251641069861,51.58971542149537,59.645718893011185,68.353262055617321,77.712344909313799,87.722967454100612],[8.4204662414513418,7.6899310424935274,7.6109355346260363,8.1834797178488685,9.4075635921620222,11.2831871575655,13.810350414059299,16.989053361643421,20.819296000317866,25.301078330082635,30.434400350937725,36.219262062883146,42.655663465918877,49.743604560044929,57.483085345261323,65.874105821568037,74.91666598896505,84.610765847452399,94.956405397030082,105.95358463769809],[9.7015200537214472,9.9575415506448373,10.86510273865855,12.424203617762586,14.634844187956945,17.497024449241625,21.01074440161663,25.176004045081957,29.992803379637607,35.461142405283582,41.581021122019877,48.3524395298465,55.775397628763436,63.849895418770693,72.575932899868292,81.953510072056204,91.982626935334423,102.66328348970298,113.99547973516187,125.97921567171107],[12.777587716407057,14.02016590921165,15.914283793106566,18.459941368091805,21.657138634167367,25.50587559133325,30.006152239589458,35.157968578935986,40.961324609372838,47.416220330900018,54.522655743517511,62.280630847225346,70.690145642023481,79.751200127911929,89.46379430489074,99.827928172959858,110.84360173211928,122.51081498236903,134.82956792370913,147.79986055613955],[17.648669229508197,19.877804118193996,22.758478697970119,26.290692968836563,30.474446930793331,35.309740583840423,40.796573927977832,46.934946963205569,53.724859689523626,61.166312106932011,69.259304215430717,78.003836015019758,87.39990750569909,97.447518687468758,108.14666956032876,119.4973601242791,131.49959037931973,144.15336032545068,157.45866996267196,171.41551929098361]],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"frame":null},{"x":[0.29999999999999999,0.27811111111111109,0.25622222222222224,0.23433333333333334,0.21244444444444444,0.19055555555555553,0.16866666666666666,0.14677777777777778,0.12488888888888888,0.10299999999999999,0.10100000000000001,0.097000000000000003,0.10349999999999999,0.10150000000000001,0.1021],"y":[9.5,8.5506666666666664,7.6013333333333328,6.6520000000000001,5.7026666666666666,4.753333333333333,3.8040000000000003,2.8546666666666667,1.9053333333333331,0.95599999999999996,0.89500000000000002,0.91200000000000003,0.89510000000000001,0.90510000000000002,0.89170000000000005],"z":[158.72801519262293,125.61538475678093,96.400588220712137,71.083625584416538,49.664496847894107,32.143202011144872,18.519741074168852,8.7941140369660147,2.9663208995363681,1.0363616618799181,1.0483015123127049,1.0690049663061476,1.0392646933675205,1.044208850863422,1.0441653258326351],"type":"scatter3d","mode":"lines","showlegend":false,"line":{"color":"#07FAFA","width":5,"type":"dashed"},"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"frame":null},{"x":[0.29999999999999999],"y":[9.5],"z":[158.72801519262293],"type":"scatter3d","mode":"markers","frame":"1","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.20000000000000001,"selected":{"opacity":1},"debounce":0},"frames":[{"name":"1","data":[{"x":[0.29999999999999999],"y":[9.5],"z":[158.72801519262293],"type":"scatter3d","mode":"markers","frame":"1","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"2","data":[{"x":[0.27811111111111109],"y":[8.5506666666666664],"z":[125.61538475678093],"type":"scatter3d","mode":"markers","frame":"2","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"3","data":[{"x":[0.25622222222222224],"y":[7.6013333333333328],"z":[96.400588220712137],"type":"scatter3d","mode":"markers","frame":"3","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"4","data":[{"x":[0.23433333333333334],"y":[6.6520000000000001],"z":[71.083625584416538],"type":"scatter3d","mode":"markers","frame":"4","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"5","data":[{"x":[0.21244444444444444],"y":[5.7026666666666666],"z":[49.664496847894107],"type":"scatter3d","mode":"markers","frame":"5","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"6","data":[{"x":[0.19055555555555553],"y":[4.753333333333333],"z":[32.143202011144872],"type":"scatter3d","mode":"markers","frame":"6","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"7","data":[{"x":[0.16866666666666666],"y":[3.8040000000000003],"z":[18.519741074168852],"type":"scatter3d","mode":"markers","frame":"7","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"8","data":[{"x":[0.14677777777777778],"y":[2.8546666666666667],"z":[8.7941140369660147],"type":"scatter3d","mode":"markers","frame":"8","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"9","data":[{"x":[0.12488888888888888],"y":[1.9053333333333331],"z":[2.9663208995363681],"type":"scatter3d","mode":"markers","frame":"9","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"10","data":[{"x":[0.10299999999999999],"y":[0.95599999999999996],"z":[1.0363616618799181],"type":"scatter3d","mode":"markers","frame":"10","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"11","data":[{"x":[0.10100000000000001],"y":[0.89500000000000002],"z":[1.0483015123127049],"type":"scatter3d","mode":"markers","frame":"11","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"12","data":[{"x":[0.097000000000000003],"y":[0.91200000000000003],"z":[1.0690049663061476],"type":"scatter3d","mode":"markers","frame":"12","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"13","data":[{"x":[0.10349999999999999],"y":[0.89510000000000001],"z":[1.0392646933675205],"type":"scatter3d","mode":"markers","frame":"13","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"14","data":[{"x":[0.10150000000000001],"y":[0.90510000000000002],"z":[1.044208850863422],"type":"scatter3d","mode":"markers","frame":"14","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]},{"name":"15","data":[{"x":[0.1021],"y":[0.89170000000000005],"z":[1.0441653258326351],"type":"scatter3d","mode":"markers","frame":"15","showlegend":false,"marker":{"color":"#70EF44","line":{"color":"rgba(44,160,44,1)"}},"error_y":{"color":"rgba(44,160,44,1)"},"error_x":{"color":"rgba(44,160,44,1)"},"line":{"color":"rgba(44,160,44,1)"},"visible":true}],"traces":[2]}],"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>
]
.pull-right[
<img src="itc_linear_files/figure-html/unnamed-chunk-13-.gif" width="450" height="350" style="display: block; margin: auto 0 auto auto;" />

- And there you have it!

]

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Multiple LR: `Tip ~ Total_bill + Size`  <h0br>
.pull-left-60[
- Pearson's correlations between variables:

- General model:

$$
\hat{Y}=X\beta\hspace{5.35cm}
$$

$$
`\begin{bmatrix}
\hat{y}_1 \\
\vdots  \\
\hat{y}_n \\
\end{bmatrix}=\begin{bmatrix}
1 & x_{11} & ... & x_{1d} \\
\vdots & \vdots & \ddots & \vdots\\
1 & x_{n1} & ... & x_{nd} \\
\end{bmatrix}\begin{bmatrix}
\beta_0 \\
\vdots  \\
\beta_d \\
\end{bmatrix}`
$$

]

.pull-right-40[
- .stress[Optimization]: `$$\hat{ \beta}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2.$$`

- .stress[Normal equation]:

`$$\hat{\beta}=(X^tX)^{-1}X^tY,$$`

- .stress[Prediction]: `$$\hat{Y}=X\hat{\beta}={\cal P}Y,$$`
with `${\cal P}=X(X^tX)^{-1}X^t$`.

- `${\cal P}=\text{proj}_{\{X_j\}}$` O.P matrix.

]
---
count:true
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Multiple LR: `Tip ~ Total_bill + Size`  <h0br>
.pull-left-70[

```r
mod1 <- lm(tip ~ total_bill + size, data = tip[mask,]); summary(mod1)
```

```

Call:
lm(formula = tip ~ total_bill + size, data = tip[mask, ])

Residuals:
    Min      1Q  Median      3Q     Max 
-2.6593 -0.6091 -0.1066  0.5016  3.4849

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.81673    0.21295   3.835  0.00017 ***
total_bill   0.07656    0.01039   7.372 4.87e-12 ***
size         0.24950    0.09213   2.708  0.00738 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.008 on 192 degrees of freedom
Multiple R-squared:  0.4112,	Adjusted R-squared:  0.405 
F-statistic: 67.03 on 2 and 192 DF,  p-value: < 2.2e-16
```
]
.pull-right-30[
<br><br><br><br><br><br>

|Model |  RMSE|
|:-----|-----:|
|1     | 1.039|
|2     | 1.070|
]

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Multiple LR: `Tip ~ All` <h0br>
.pull-left-70[<hbr>

```

Call:
lm(formula = tip ~ ., data = tip[mask, ])

Residuals:
    Min      1Q  Median      3Q     Max 
-2.5336 -0.5943 -0.1326  0.5160  3.5293

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.852677   0.298846   2.853  0.00482 ** 
total_bill   0.079678   0.010909   7.304 7.94e-12 ***
sexMale     -0.044752   0.156404  -0.286  0.77509    
smokerYes   -0.075187   0.162533  -0.463  0.64419    
day.L        0.035309   0.390170   0.090  0.92799    
day.Q       -0.003551   0.248848  -0.014  0.98863    
day.C        0.166458   0.200445   0.830  0.40735    
timeLunch    0.157262   0.496369   0.317  0.75173    
size         0.228495   0.096436   2.369  0.01884 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.018 on 186 degrees of freedom
Multiple R-squared:  0.4181,	Adjusted R-squared:  0.3931 
F-statistic: 16.71 on 8 and 186 DF,  p-value: < 2.2e-16
```
]

|Model |  RMSE|
|:-----|-----:|
|1     | 1.039|
|2     | 1.070|
|3     | 1.092|
]

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Regularization <h0br>

.pull-left[
<img src="itc_linear_files/figure-html/line0-1.png" style="display: block; margin: auto;" />

]

--
.pull-right[
- Regularization formulation:
`$$(\hat{a},\hat{b})=\arg\min_{(a,b)\in\mathbb{R}^2}\text{MSE}+\lambda{\cal R}(a,b)$$`
where `${\cal R}$` is a .stress[regularization] term: 
  - `${\cal R}(a,b)=\|(a,b)\|_1=|a|+|b|$`
  - `${\cal R}(a,b)=\|(a,b)\|_2^2=a^2+b^2.$`
  
- `$\lambda$` to be tuned, 
$$
`\begin{aligned}
\text{Large } \lambda\hspace{1.8cm} \\
\Updownarrow \hspace{2.5cm}\\
\text{Strong regularization } \\
\Updownarrow \hspace{2.5cm}\\
\text{Small parameters} \hspace{0.5cm}
\end{aligned}`
$$
]

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; `$\ell_2$` regularization <h0br>

<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-bbc676163c4b09796f4d" style="width:648px;height:396px;"></div>
<script type="application/json" data-for="htmlwidget-bbc676163c4b09796f4d">{"x":{"visdat":{"ee823e1aae":["function () ","plotlyVisDat"]},"cur_data":"ee823e1aae","attrs":{"ee823e1aae":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"inherit":true},"ee823e1aae.1":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"scatter3d","mode":"lines","frame":{},"showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"camera":{"eye":{"x":0,"y":-1.25,"z":0.69999999999999996}},"xaxis":{"title":"a"},"yaxis":{"title":"b"},"zaxis":{"title":"Loss"}},"hovermode":"closest","showlegend":false,"xaxis":{"range":[-0.22500000000000001,0.32500000000000001]},"yaxis":{"range":[-8.9000000000000004,10.9]},"sliders":[{"currentvalue":{"prefix":"lambda: ","xanchor":"right","font":{"size":16,"color":"rgba(204,204,204,1)"}},"steps":[{"method":"animate","args":[["0.1"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"0.1","value":"0.1"},{"method":"animate","args":[["0.75"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"0.75","value":"0.75"},{"method":"animate","args":[["3"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"3","value":"3"},{"method":"animate","args":[["5"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"5","value":"5"}],"visible":true,"pad":{"t":40}}],"updatemenus":[{"type":"buttons","direction":"right","showactive":false,"y":0,"x":0,"yanchor":"top","xanchor":"right","pad":{"t":60,"r":5},"buttons":[{"label":"Play","method":"animate","args":[null,{"fromcurrent":true,"mode":"immediate","transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true}}]}]}],"legend":{"yanchor":"top","y":0.5}},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false},"data":[{"colorbar":{"title":"Loss<br />z","ticklen":2,"len":0.5,"lenmode":"fraction","y":1,"yanchor":"top"},"colorscale":[["0","rgba(68,1,84,1)"],["0.0416666666666667","rgba(70,19,97,1)"],["0.0833333333333333","rgba(72,32,111,1)"],["0.125","rgba(71,45,122,1)"],["0.166666666666667","rgba(68,58,128,1)"],["0.208333333333333","rgba(64,70,135,1)"],["0.25","rgba(60,82,138,1)"],["0.291666666666667","rgba(56,93,140,1)"],["0.333333333333333","rgba(49,104,142,1)"],["0.375","rgba(46,114,142,1)"],["0.416666666666667","rgba(42,123,142,1)"],["0.458333333333333","rgba(38,133,141,1)"],["0.5","rgba(37,144,140,1)"],["0.541666666666667","rgba(33,154,138,1)"],["0.583333333333333","rgba(39,164,133,1)"],["0.625","rgba(47,174,127,1)"],["0.666666666666667","rgba(53,183,121,1)"],["0.708333333333333","rgba(79,191,110,1)"],["0.75","rgba(98,199,98,1)"],["0.791666666666667","rgba(119,207,85,1)"],["0.833333333333333","rgba(147,214,70,1)"],["0.875","rgba(172,220,52,1)"],["0.916666666666667","rgba(199,225,42,1)"],["0.958333333333333","rgba(226,228,40,1)"],["1","rgba(253,231,37,1)"]],"showscale":true,"x":[-0.20000000000000001,-0.1736842105263158,-0.14736842105263159,-0.12105263157894738,-0.094736842105263175,-0.068421052631578966,-0.042105263157894757,-0.015789473684210548,0.010526315789473661,0.03684210526315787,0.06315789473684208,0.089473684210526316,0.1157894736842105,0.14210526315789468,0.16842105263157892,0.19473684210526315,0.22105263157894733,0.24736842105263152,0.27368421052631575,0.29999999999999999],"y":[-8,-7.0526315789473681,-6.1052631578947372,-5.1578947368421053,-4.2105263157894743,-3.2631578947368425,-2.3157894736842106,-1.3684210526315796,-0.42105263157894779,0.52631578947368318,1.473684210526315,2.4210526315789469,3.3684210526315788,4.3157894736842106,5.2631578947368407,6.2105263157894726,7.1578947368421044,8.1052631578947363,9.0526315789473664,10],"z":[[232.04548890163935,215.53004656858226,199.66614392661552,184.45378097573908,169.89295771595297,155.98367414725718,142.72593026965171,130.11972608313656,118.16506158771173,106.86193678337725,96.210351670133065,86.210306247979204,76.861800516915679,68.164834476942488,60.11940812805959,52.725521470267019,45.98317450356479,39.892367227952874,34.45309964343128,29.665371750000002],[204.60632110726124,189.07743547008536,174.20008952399982,159.97428326900459,146.40001670509969,133.47728983228509,121.20610265056084,109.58645515992689,98.618347360383268,88.301779251929986,78.63675083456701,69.623262108294355,61.261313073112035,53.550903729020035,46.492034076018349,40.084704114106984,34.32891384328596,29.22466326355525,24.771952374914857,20.970781177364788],[178.96216716329869,164.41983822200402,150.52904897179965,137.28979941268562,124.70208954466193,112.76591936772854,101.48128888188549,90.848198087132758,80.866646983470332,71.536635570898241,62.858163849416478,54.831231819025028,47.455839479723913,40.731986831513119,34.659673874392638,29.238900608362478,24.469667033422652,20.351973149573148,16.885818956813957,14.071204455145091],[155.11302706975161,141.55725482433814,128.65302227001499,116.40032940678216,104.79917623463967,93.849562753587492,83.551488963625644,73.904954864754103,64.909960456972897,56.566505740282011,48.874590714681446,41.834215380171202,35.445379736751292,29.708083784421703,24.622327523182424,20.188110953033469,16.405434073974849,13.274296886006546,10.794699389128562,8.9666415833409019],[133.05890082662006,120.4896852770878,108.57200941864586,97.30587325129423,86.691276775032946,76.728219989861969,67.416702895781313,58.756725492790991,50.748287780890983,43.391389760081303,36.686031430361943,30.632212791732901,25.229933844194189,20.479194587745802,16.379995022387732,12.93233514811998,10.136214964942562,7.9916344728554636,6.4985936718586848,5.6570925619522292],[112.79978843390401,101.21712958025296,90.286010417692211,80.006430946221798,70.378391165841705,61.401891076551941,53.076930678352497,45.403509971243366,38.381628955224571,32.011287630296088,26.292485996457934,21.225224053710097,16.809501802052594,13.04531924148541,9.932676372008542,7.4715731936219996,5.662009706325783,4.503985910119888,3.997501805004315,4.1425573909790652],[94.335689891603479,83.739587733833616,73.795025267154088,64.50200249156488,55.860519407065986,47.870576013657427,40.532172311339181,33.845308300111263,27.809983979973669,22.426199350926396,17.693954412969443,13.613249166102811,10.18408361032651,7.4064577456405312,5.280371572044869,3.8058250895395309,2.9828182981245184,2.8113511977998269,3.2914237885654591,4.4230360704214151],[77.666605199718461,68.057059737829817,59.09905396703148,50.792587887323478,43.137661498705796,36.134274801178435,29.782427794741395,24.082120479394678,19.033352855138286,14.636124921972216,10.890436679896471,7.7962881289110433,5.3536792690159443,3.5626101002111676,2.4230806224967103,1.9350908358725769,2.0986407403387668,2.913730335895278,4.3803596225421151,6.4985286002792755],[62.792534358248957,54.169545592241505,46.19809651732438,38.878187133497583,32.2098174407611,26.192987439114944,20.827697128559109,16.113946509093598,12.051735580718411,8.6410643434335466,5.8819327972390036,3.7743409421347818,2.3182887781208867,1.5137763051973132,1.3608035233640614,1.8593704326211338,3.0094770329685274,4.8111233244062417,7.2643093069342841,10.36903498055265],[49.713477367194969,42.077045297068722,35.092152918032802,28.758800230087203,23.076987233231929,18.046713927466975,13.667980312792345,9.9407863892080375,6.8651321567140524,4.4410176153103906,2.6684427649970521,1.5474076057740354,1.077912137641343,1.2599563605989719,2.0935402746469247,3.5786638797852017,5.7153271760137976,8.5035301633327158,11.943272841741962,16.034555211241532],[38.429434226556481,31.779558852311439,25.781223169156725,20.434427177092328,15.739170876118258,11.69545426623451,8.3032773474410853,5.5626401197379822,3.473542583125202,2.0359847376027451,1.2499665831706106,1.1154881198287994,1.6325493475773105,2.8011502664161427,4.6212908763453013,7.0929711773647837,10.216191169474584,13.990950852674704,18.417250226965155,23.495089292345934],[28.940404936333504,23.277086257969671,18.265307270696159,13.905067974512971,10.196368369420103,7.1392084554175597,4.7335882325053387,2.9795077006834405,1.8769668599518652,1.4259657103106125,1.6265042517596828,2.4785824842990771,3.9822004079287914,6.1373580226488276,8.944055328459191,12.40229232535988,16.512069013350882,21.273385392432207,26.686241462603867,32.750637223865844],[21.246389496526046,16.569627514043415,12.544405222651109,9.1707226223491229,6.4485797131374625,4.3779764950161226,2.9589129679851061,2.1913891320444123,2.0754049871940414,2.6109605334339938,3.7980557707642681,5.636690699184868,8.1268653186957867,11.268579629297026,15.061833630988595,19.506627323770488,24.602960707642694,30.350833782605225,36.75024654865809,43.801199005801273],[15.347387907134101,11.657182620532675,8.6185170250215712,6.231391120600791,4.4958049072703341,3.4117583850301991,2.979251553880387,3.1982844138208981,4.0688569648517321,5.590969206972888,7.7646211401843672,10.589812764486172,14.066544079878295,18.194815086360737,22.974625783933512,28.405976172596613,34.488866252350022,41.223296023193754,48.609265485127821,56.646774638152216],[11.243400168157674,8.5397515774374497,6.4876426778075498,5.0870734692679722,4.3380439518187179,4.2405541254597861,4.7946039901911766,6.0001935460128903,7.8573227929249265,10.365991730927286,13.526200360019967,17.337948680202977,21.801236691476301,26.916064393839946,32.682431787293922,39.100338871838225,46.169785647472835,53.890772114197773,62.263298272013046,71.287364120918639],[8.9344262795967513,7.2173343847577316,6.151782181009037,5.7377696683506629,5.975296846782614,6.8643637163048856,8.4049702769174814,10.597116528620399,13.44080247141364,16.936028105297204,21.082793430271092,25.881098446335304,31.33094315348983,37.43232755173468,44.185251641069861,51.58971542149537,59.645718893011185,68.353262055617321,77.712344909313799,87.722967454100612],[8.4204662414513418,7.6899310424935274,7.6109355346260363,8.1834797178488685,9.4075635921620222,11.2831871575655,13.810350414059299,16.989053361643421,20.819296000317866,25.301078330082635,30.434400350937725,36.219262062883146,42.655663465918877,49.743604560044929,57.483085345261323,65.874105821568037,74.91666598896505,84.610765847452399,94.956405397030082,105.95358463769809],[9.7015200537214472,9.9575415506448373,10.86510273865855,12.424203617762586,14.634844187956945,17.497024449241625,21.01074440161663,25.176004045081957,29.992803379637607,35.461142405283582,41.581021122019877,48.3524395298465,55.775397628763436,63.849895418770693,72.575932899868292,81.953510072056204,91.982626935334423,102.66328348970298,113.99547973516187,125.97921567171107],[12.777587716407057,14.02016590921165,15.914283793106566,18.459941368091805,21.657138634167367,25.50587559133325,30.006152239589458,35.157968578935986,40.961324609372838,47.416220330900018,54.522655743517511,62.280630847225346,70.690145642023481,79.751200127911929,89.46379430489074,99.827928172959858,110.84360173211928,122.51081498236903,134.82956792370913,147.79986055613955],[17.648669229508197,19.877804118193996,22.758478697970119,26.290692968836563,30.474446930793331,35.309740583840423,40.796573927977832,46.934946963205569,53.724859689523626,61.166312106932011,69.259304215430717,78.003836015019758,87.39990750569909,97.447518687468758,108.14666956032876,119.4973601242791,131.49959037931973,144.15336032545068,157.45866996267196,171.41551929098361]],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"frame":null},{"x":[-0.20000000000000001,-0.19532411114201734,-0.18151508393419138,-0.15921861314112876,-0.12947725695636553,-0.093681688139958016,-0.053505667705844175,-0.010827781717083498,0.032356399310552973,0.074027631067982921,0.11223741307247651,0.14519909838462616,0.17137143523351786,0.1895306342365605,0.19882759143087192,0.19882759143087192,0.1895306342365605,0.17137143523351789,0.14519909838462616,0.11223741307247651,0.074027631067982824,0.032356399310552973,-0.010827781717083453,-0.053505667705844175,-0.093681688139958016,-0.12947725695636544,-0.15921861314112876,-0.18151508393419138,-0.19532411114201731,-0.20000000000000001],"y":[-2.9390552491737056e-16,-0.51592905650645826,-1.0077338437446357,-1.4524181164650365,-1.8291889323063275,-2.1204289066704551,-2.3125199820461351,-2.3964801932426973,-2.368383654099663,-2.2295441275602994,-1.9864535955765372,-1.6504787012482161,-1.2373292572248522,-0.76632367232635179,-0.25948564421746079,0.25948564421746079,0.76632367232635179,1.2373292572248515,1.6504787012482161,1.9864535955765372,2.2295441275602998,2.368383654099663,2.3964801932426978,2.3125199820461351,2.1204289066704551,1.8291889323063284,1.4524181164650365,1.0077338437446357,0.51592905650645937,2.9390552491737056e-16],"z":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"0.1","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.20000000000000001,"selected":{"opacity":1},"debounce":0},"frames":[{"name":"0.1","data":[{"x":[-0.20000000000000001,-0.19532411114201734,-0.18151508393419138,-0.15921861314112876,-0.12947725695636553,-0.093681688139958016,-0.053505667705844175,-0.010827781717083498,0.032356399310552973,0.074027631067982921,0.11223741307247651,0.14519909838462616,0.17137143523351786,0.1895306342365605,0.19882759143087192,0.19882759143087192,0.1895306342365605,0.17137143523351789,0.14519909838462616,0.11223741307247651,0.074027631067982824,0.032356399310552973,-0.010827781717083453,-0.053505667705844175,-0.093681688139958016,-0.12947725695636544,-0.15921861314112876,-0.18151508393419138,-0.19532411114201731,-0.20000000000000001],"y":[-2.9390552491737056e-16,-0.51592905650645826,-1.0077338437446357,-1.4524181164650365,-1.8291889323063275,-2.1204289066704551,-2.3125199820461351,-2.3964801932426973,-2.368383654099663,-2.2295441275602994,-1.9864535955765372,-1.6504787012482161,-1.2373292572248522,-0.76632367232635179,-0.25948564421746079,0.25948564421746079,0.76632367232635179,1.2373292572248515,1.6504787012482161,1.9864535955765372,2.2295441275602998,2.368383654099663,2.3964801932426978,2.3125199820461351,2.1204289066704551,1.8291889323063284,1.4524181164650365,1.0077338437446357,0.51592905650645937,2.9390552491737056e-16],"z":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"0.1","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"traces":[1]},{"name":"0.75","data":[{"x":[-0.10000000000000001,-0.097662055571008669,-0.090757541967095692,-0.079609306570564378,-0.064738628478182764,-0.046840844069979008,-0.026752833852922087,-0.0054138908585417492,0.016178199655276487,0.037013815533991461,0.056118706536238253,0.072599549192313081,0.08568571761675893,0.094765317118280248,0.09941379571543596,0.09941379571543596,0.094765317118280248,0.085685717616758944,0.072599549192313081,0.056118706536238253,0.037013815533991412,0.016178199655276487,-0.0054138908585417266,-0.026752833852922087,-0.046840844069979008,-0.064738628478182722,-0.079609306570564378,-0.090757541967095692,-0.097662055571008655,-0.10000000000000001],"y":[-1.4695276245868528e-16,-0.25796452825322913,-0.50386692187231785,-0.72620905823251825,-0.91459446615316375,-1.0602144533352276,-1.1562599910230675,-1.1982400966213487,-1.1841918270498315,-1.1147720637801497,-0.99322679778826861,-0.82523935062410803,-0.6186646286124261,-0.3831618361631759,-0.12974282210873039,0.12974282210873039,0.3831618361631759,0.61866462861242577,0.82523935062410803,0.99322679778826861,1.1147720637801499,1.1841918270498315,1.1982400966213489,1.1562599910230675,1.0602144533352276,0.9145944661531642,0.72620905823251825,0.50386692187231785,0.25796452825322969,1.4695276245868528e-16],"z":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"0.75","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"traces":[1]},{"name":"3","data":[{"x":[-0.050000000000000003,-0.048831027785504334,-0.045378770983547846,-0.039804653285282189,-0.032369314239091382,-0.023420422034989504,-0.013376416926461044,-0.0027069454292708746,0.0080890998276382433,0.01850690776699573,0.028059353268119126,0.036299774596156541,0.042842858808379465,0.047382658559140124,0.04970689785771798,0.04970689785771798,0.047382658559140124,0.042842858808379472,0.036299774596156541,0.028059353268119126,0.018506907766995706,0.0080890998276382433,-0.0027069454292708633,-0.013376416926461044,-0.023420422034989504,-0.032369314239091361,-0.039804653285282189,-0.045378770983547846,-0.048831027785504327,-0.050000000000000003],"y":[-7.347638122934264e-17,-0.12898226412661457,-0.25193346093615893,-0.36310452911625912,-0.45729723307658188,-0.53010722666761378,-0.57812999551153377,-0.59912004831067434,-0.59209591352491575,-0.55738603189007485,-0.49661339889413431,-0.41261967531205401,-0.30933231430621305,-0.19158091808158795,-0.064871411054365197,0.064871411054365197,0.19158091808158795,0.30933231430621289,0.41261967531205401,0.49661339889413431,0.55738603189007496,0.59209591352491575,0.59912004831067445,0.57812999551153377,0.53010722666761378,0.4572972330765821,0.36310452911625912,0.25193346093615893,0.12898226412661484,7.347638122934264e-17],"z":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"3","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"traces":[1]},{"name":"5","data":[{"x":[-0.01,-0.0097662055571008672,-0.0090757541967095689,-0.0079609306570564385,-0.006473862847818276,-0.0046840844069979005,-0.0026752833852922088,-0.00054138908585417487,0.0016178199655276484,0.0037013815533991456,0.0056118706536238249,0.0072599549192313081,0.008568571761675893,0.0094765317118280252,0.0099413795715435956,0.0099413795715435956,0.0094765317118280252,0.0085685717616758948,0.0072599549192313081,0.0056118706536238249,0.0037013815533991413,0.0016178199655276484,-0.0005413890858541726,-0.0026752833852922088,-0.0046840844069979005,-0.0064738628478182726,-0.0079609306570564385,-0.0090757541967095689,-0.0097662055571008655,-0.01],"y":[-1.4695276245868526e-17,-0.025796452825322912,-0.050386692187231782,-0.072620905823251813,-0.091459446615316381,-0.10602144533352276,-0.11562599910230675,-0.11982400966213487,-0.11841918270498314,-0.11147720637801498,-0.099322679778826861,-0.082523935062410805,-0.061866462861242609,-0.038316183616317598,-0.012974282210873038,0.012974282210873038,0.038316183616317598,0.061866462861242567,0.082523935062410805,0.099322679778826861,0.11147720637801498,0.11841918270498314,0.11982400966213487,0.11562599910230675,0.10602144533352276,0.091459446615316423,0.072620905823251813,0.050386692187231782,0.025796452825322967,1.4695276245868526e-17],"z":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"5","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"traces":[1]}],"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

`$$(\hat{a},\hat{b})=\arg\min_{(a,b)\in\mathbb{R}^2}\sum_{i=1}^n(y_i-ax_i+b)^2+\lambda \|(a,b)\|_2^2$$`

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; `$\ell_1$` regularization <h0br>

<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-9bd4b2449e6193e7f43d" style="width:648px;height:396px;"></div>
<script type="application/json" data-for="htmlwidget-9bd4b2449e6193e7f43d">{"x":{"visdat":{"ee870761481":["function () ","plotlyVisDat"]},"cur_data":"ee870761481","attrs":{"ee870761481":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"inherit":true},"ee870761481.1":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"scatter3d","mode":"lines","frame":{},"showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"camera":{"eye":{"x":0,"y":-1.25,"z":0.69999999999999996}},"xaxis":{"title":"a"},"yaxis":{"title":"b"},"zaxis":{"title":"Loss"}},"hovermode":"closest","showlegend":false,"xaxis":{"range":[-0.22500000000000001,0.32500000000000001]},"yaxis":{"range":[-8.9000000000000004,10.9]},"sliders":[{"currentvalue":{"prefix":"lambda: ","xanchor":"right","font":{"size":16,"color":"rgba(204,204,204,1)"}},"steps":[{"method":"animate","args":[["0.1"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"0.1","value":"0.1"},{"method":"animate","args":[["0.75"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"0.75","value":"0.75"},{"method":"animate","args":[["3"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"3","value":"3"},{"method":"animate","args":[["5"],{"transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true},"mode":"immediate"}],"label":"5","value":"5"}],"visible":true,"pad":{"t":40}}],"updatemenus":[{"type":"buttons","direction":"right","showactive":false,"y":0,"x":0,"yanchor":"top","xanchor":"right","pad":{"t":60,"r":5},"buttons":[{"label":"Play","method":"animate","args":[null,{"fromcurrent":true,"mode":"immediate","transition":{"duration":500,"easing":"linear"},"frame":{"duration":500,"redraw":true}}]}]}],"legend":{"yanchor":"top","y":0.5}},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false},"data":[{"colorbar":{"title":"Loss<br />z","ticklen":2,"len":0.5,"lenmode":"fraction","y":1,"yanchor":"top"},"colorscale":[["0","rgba(68,1,84,1)"],["0.0416666666666667","rgba(70,19,97,1)"],["0.0833333333333333","rgba(72,32,111,1)"],["0.125","rgba(71,45,122,1)"],["0.166666666666667","rgba(68,58,128,1)"],["0.208333333333333","rgba(64,70,135,1)"],["0.25","rgba(60,82,138,1)"],["0.291666666666667","rgba(56,93,140,1)"],["0.333333333333333","rgba(49,104,142,1)"],["0.375","rgba(46,114,142,1)"],["0.416666666666667","rgba(42,123,142,1)"],["0.458333333333333","rgba(38,133,141,1)"],["0.5","rgba(37,144,140,1)"],["0.541666666666667","rgba(33,154,138,1)"],["0.583333333333333","rgba(39,164,133,1)"],["0.625","rgba(47,174,127,1)"],["0.666666666666667","rgba(53,183,121,1)"],["0.708333333333333","rgba(79,191,110,1)"],["0.75","rgba(98,199,98,1)"],["0.791666666666667","rgba(119,207,85,1)"],["0.833333333333333","rgba(147,214,70,1)"],["0.875","rgba(172,220,52,1)"],["0.916666666666667","rgba(199,225,42,1)"],["0.958333333333333","rgba(226,228,40,1)"],["1","rgba(253,231,37,1)"]],"showscale":true,"x":[-0.20000000000000001,-0.1736842105263158,-0.14736842105263159,-0.12105263157894738,-0.094736842105263175,-0.068421052631578966,-0.042105263157894757,-0.015789473684210548,0.010526315789473661,0.03684210526315787,0.06315789473684208,0.089473684210526316,0.1157894736842105,0.14210526315789468,0.16842105263157892,0.19473684210526315,0.22105263157894733,0.24736842105263152,0.27368421052631575,0.29999999999999999],"y":[-8,-7.0526315789473681,-6.1052631578947372,-5.1578947368421053,-4.2105263157894743,-3.2631578947368425,-2.3157894736842106,-1.3684210526315796,-0.42105263157894779,0.52631578947368318,1.473684210526315,2.4210526315789469,3.3684210526315788,4.3157894736842106,5.2631578947368407,6.2105263157894726,7.1578947368421044,8.1052631578947363,9.0526315789473664,10],"z":[[232.04548890163935,215.53004656858226,199.66614392661552,184.45378097573908,169.89295771595297,155.98367414725718,142.72593026965171,130.11972608313656,118.16506158771173,106.86193678337725,96.210351670133065,86.210306247979204,76.861800516915679,68.164834476942488,60.11940812805959,52.725521470267019,45.98317450356479,39.892367227952874,34.45309964343128,29.665371750000002],[204.60632110726124,189.07743547008536,174.20008952399982,159.97428326900459,146.40001670509969,133.47728983228509,121.20610265056084,109.58645515992689,98.618347360383268,88.301779251929986,78.63675083456701,69.623262108294355,61.261313073112035,53.550903729020035,46.492034076018349,40.084704114106984,34.32891384328596,29.22466326355525,24.771952374914857,20.970781177364788],[178.96216716329869,164.41983822200402,150.52904897179965,137.28979941268562,124.70208954466193,112.76591936772854,101.48128888188549,90.848198087132758,80.866646983470332,71.536635570898241,62.858163849416478,54.831231819025028,47.455839479723913,40.731986831513119,34.659673874392638,29.238900608362478,24.469667033422652,20.351973149573148,16.885818956813957,14.071204455145091],[155.11302706975161,141.55725482433814,128.65302227001499,116.40032940678216,104.79917623463967,93.849562753587492,83.551488963625644,73.904954864754103,64.909960456972897,56.566505740282011,48.874590714681446,41.834215380171202,35.445379736751292,29.708083784421703,24.622327523182424,20.188110953033469,16.405434073974849,13.274296886006546,10.794699389128562,8.9666415833409019],[133.05890082662006,120.4896852770878,108.57200941864586,97.30587325129423,86.691276775032946,76.728219989861969,67.416702895781313,58.756725492790991,50.748287780890983,43.391389760081303,36.686031430361943,30.632212791732901,25.229933844194189,20.479194587745802,16.379995022387732,12.93233514811998,10.136214964942562,7.9916344728554636,6.4985936718586848,5.6570925619522292],[112.79978843390401,101.21712958025296,90.286010417692211,80.006430946221798,70.378391165841705,61.401891076551941,53.076930678352497,45.403509971243366,38.381628955224571,32.011287630296088,26.292485996457934,21.225224053710097,16.809501802052594,13.04531924148541,9.932676372008542,7.4715731936219996,5.662009706325783,4.503985910119888,3.997501805004315,4.1425573909790652],[94.335689891603479,83.739587733833616,73.795025267154088,64.50200249156488,55.860519407065986,47.870576013657427,40.532172311339181,33.845308300111263,27.809983979973669,22.426199350926396,17.693954412969443,13.613249166102811,10.18408361032651,7.4064577456405312,5.280371572044869,3.8058250895395309,2.9828182981245184,2.8113511977998269,3.2914237885654591,4.4230360704214151],[77.666605199718461,68.057059737829817,59.09905396703148,50.792587887323478,43.137661498705796,36.134274801178435,29.782427794741395,24.082120479394678,19.033352855138286,14.636124921972216,10.890436679896471,7.7962881289110433,5.3536792690159443,3.5626101002111676,2.4230806224967103,1.9350908358725769,2.0986407403387668,2.913730335895278,4.3803596225421151,6.4985286002792755],[62.792534358248957,54.169545592241505,46.19809651732438,38.878187133497583,32.2098174407611,26.192987439114944,20.827697128559109,16.113946509093598,12.051735580718411,8.6410643434335466,5.8819327972390036,3.7743409421347818,2.3182887781208867,1.5137763051973132,1.3608035233640614,1.8593704326211338,3.0094770329685274,4.8111233244062417,7.2643093069342841,10.36903498055265],[49.713477367194969,42.077045297068722,35.092152918032802,28.758800230087203,23.076987233231929,18.046713927466975,13.667980312792345,9.9407863892080375,6.8651321567140524,4.4410176153103906,2.6684427649970521,1.5474076057740354,1.077912137641343,1.2599563605989719,2.0935402746469247,3.5786638797852017,5.7153271760137976,8.5035301633327158,11.943272841741962,16.034555211241532],[38.429434226556481,31.779558852311439,25.781223169156725,20.434427177092328,15.739170876118258,11.69545426623451,8.3032773474410853,5.5626401197379822,3.473542583125202,2.0359847376027451,1.2499665831706106,1.1154881198287994,1.6325493475773105,2.8011502664161427,4.6212908763453013,7.0929711773647837,10.216191169474584,13.990950852674704,18.417250226965155,23.495089292345934],[28.940404936333504,23.277086257969671,18.265307270696159,13.905067974512971,10.196368369420103,7.1392084554175597,4.7335882325053387,2.9795077006834405,1.8769668599518652,1.4259657103106125,1.6265042517596828,2.4785824842990771,3.9822004079287914,6.1373580226488276,8.944055328459191,12.40229232535988,16.512069013350882,21.273385392432207,26.686241462603867,32.750637223865844],[21.246389496526046,16.569627514043415,12.544405222651109,9.1707226223491229,6.4485797131374625,4.3779764950161226,2.9589129679851061,2.1913891320444123,2.0754049871940414,2.6109605334339938,3.7980557707642681,5.636690699184868,8.1268653186957867,11.268579629297026,15.061833630988595,19.506627323770488,24.602960707642694,30.350833782605225,36.75024654865809,43.801199005801273],[15.347387907134101,11.657182620532675,8.6185170250215712,6.231391120600791,4.4958049072703341,3.4117583850301991,2.979251553880387,3.1982844138208981,4.0688569648517321,5.590969206972888,7.7646211401843672,10.589812764486172,14.066544079878295,18.194815086360737,22.974625783933512,28.405976172596613,34.488866252350022,41.223296023193754,48.609265485127821,56.646774638152216],[11.243400168157674,8.5397515774374497,6.4876426778075498,5.0870734692679722,4.3380439518187179,4.2405541254597861,4.7946039901911766,6.0001935460128903,7.8573227929249265,10.365991730927286,13.526200360019967,17.337948680202977,21.801236691476301,26.916064393839946,32.682431787293922,39.100338871838225,46.169785647472835,53.890772114197773,62.263298272013046,71.287364120918639],[8.9344262795967513,7.2173343847577316,6.151782181009037,5.7377696683506629,5.975296846782614,6.8643637163048856,8.4049702769174814,10.597116528620399,13.44080247141364,16.936028105297204,21.082793430271092,25.881098446335304,31.33094315348983,37.43232755173468,44.185251641069861,51.58971542149537,59.645718893011185,68.353262055617321,77.712344909313799,87.722967454100612],[8.4204662414513418,7.6899310424935274,7.6109355346260363,8.1834797178488685,9.4075635921620222,11.2831871575655,13.810350414059299,16.989053361643421,20.819296000317866,25.301078330082635,30.434400350937725,36.219262062883146,42.655663465918877,49.743604560044929,57.483085345261323,65.874105821568037,74.91666598896505,84.610765847452399,94.956405397030082,105.95358463769809],[9.7015200537214472,9.9575415506448373,10.86510273865855,12.424203617762586,14.634844187956945,17.497024449241625,21.01074440161663,25.176004045081957,29.992803379637607,35.461142405283582,41.581021122019877,48.3524395298465,55.775397628763436,63.849895418770693,72.575932899868292,81.953510072056204,91.982626935334423,102.66328348970298,113.99547973516187,125.97921567171107],[12.777587716407057,14.02016590921165,15.914283793106566,18.459941368091805,21.657138634167367,25.50587559133325,30.006152239589458,35.157968578935986,40.961324609372838,47.416220330900018,54.522655743517511,62.280630847225346,70.690145642023481,79.751200127911929,89.46379430489074,99.827928172959858,110.84360173211928,122.51081498236903,134.82956792370913,147.79986055613955],[17.648669229508197,19.877804118193996,22.758478697970119,26.290692968836563,30.474446930793331,35.309740583840423,40.796573927977832,46.934946963205569,53.724859689523626,61.166312106932011,69.259304215430717,78.003836015019758,87.39990750569909,97.447518687468758,108.14666956032876,119.4973601242791,131.49959037931973,144.15336032545068,157.45866996267196,171.41551929098361]],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"frame":null},{"x":[-0.20000000000000001,0,0.20000000000000001,0,-0.20000000000000001],"y":[0,-2.4000000000000004,0,2.4000000000000004,0],"z":[0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"0.1","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.20000000000000001,"selected":{"opacity":1},"debounce":0},"frames":[{"name":"0.1","data":[{"x":[-0.20000000000000001,0,0.20000000000000001,0,-0.20000000000000001],"y":[0,-2.4000000000000004,0,2.4000000000000004,0],"z":[0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"0.1","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"traces":[1]},{"name":"0.75","data":[{"x":[-0.10000000000000001,0,0.10000000000000001,0,-0.10000000000000001],"y":[0,-1.2000000000000002,0,1.2000000000000002,0],"z":[0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"0.75","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"traces":[1]},{"name":"3","data":[{"x":[-0.050000000000000003,0,0.050000000000000003,0,-0.050000000000000003],"y":[0,-0.60000000000000009,0,0.60000000000000009,0],"z":[0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"3","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"traces":[1]},{"name":"5","data":[{"x":[-0.01,0,0.01,0,-0.01],"y":[0,-0.12,0,0.12,0],"z":[0,0,0,0,0],"type":"scatter3d","mode":"lines","frame":"5","showlegend":false,"line":{"color":"#F93228","size":20},"opacity":0.59999999999999998,"marker":{"color":"rgba(255,127,14,1)","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"visible":true}],"traces":[1]}],"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

`$$(\hat{a},\hat{b})=\arg\min_{(a,b)\in\mathbb{R}^2}\sum_{i=1}^n(y_i-ax_i+b)^2+\lambda \|(a,b)\|_1$$`

---
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Ridge regression ( `$\ell_2$`-norm ) <h0br>

`$$\hat{\beta}^{\text{ridge}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda\|\beta\|_2^2$$`

- Solution:

`$$\hat{\beta}^{\text{ridge}}=(X^tX+\lambda I)^{-1}X^tY$$`

]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Ridge regression ( `$\ell_2$`-norm ) <h0br>
.pull-left[
- In general, for any `$\lambda>0$`:

`$$\hat{\beta}^{\text{ridge}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda\|\beta\|_2^2$$`

- Solution:

`$$\hat{\beta}^{\text{ridge}}=(X^tX+\lambda I)^{-1}X^tY$$`

- .stress[K-fold cross validation]:

- `For each` `$\lambda\in\{\lambda_1, \lambda_2, ...,\lambda_J\}$`

]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Ridge regression ( `$\ell_2$`-norm ) <h0br>
.pull-left[
- In general, for any `$\lambda>0$`:

`$$\hat{\beta}^{\text{ridge}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda\|\beta\|_2^2$$`

- Solution:

`$$\hat{\beta}^{\text{ridge}}=(X^tX+\lambda I)^{-1}X^tY$$`

- .stress[K-fold cross validation]:

- `For each` `$\lambda\in\{\lambda_1, \lambda_2, ...,\lambda_J\}$`

]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Ridge regression ( `$\ell_2$`-norm ) <h0br>
.pull-left[
- In general, for any `$\lambda>0$`:

`$$\hat{\beta}^{\text{ridge}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda\|\beta\|_2^2$$`

- Solution:

`$$\hat{\beta}^{\text{ridge}}=(X^tX+\lambda I)^{-1}X^tY$$`

- .stress[K-fold cross validation]:

- `For each` `$\lambda\in\{\lambda_1, \lambda_2, ...,\lambda_J\}$`

]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Ridge regression ( `$\ell_2$`-norm ) <h0br>
.pull-left[
- In general, for any `$\lambda>0$`:

`$$\hat{\beta}^{\text{ridge}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda\|\beta\|_2^2$$`

- Solution:

`$$\hat{\beta}^{\text{ridge}}=(X^tX+\lambda I)^{-1}X^tY$$`

- .stress[K-fold cross validation]:

- `For each` `$\lambda\in\{\lambda_1, \lambda_2, ...,\lambda_J\}$`

]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Ridge regression ( `$\ell_2$`-norm ) <h0br>
.pull-left[
- In general, for any `$\lambda>0$`:

`$$\hat{\beta}^{\text{ridge}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda\|\beta\|_2^2$$`

- Solution:

`$$\hat{\beta}^{\text{ridge}}=(X^tX+\lambda I)^{-1}X^tY$$`

- .stress[K-fold cross validation]:

- `For each` `$\lambda\in\{\lambda_1, \lambda_2, ...,\lambda_J\}$`

]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Ridge regression ( `$\ell_2$`-norm ) <h0br>
.pull-left[
- In general, for any `$\lambda>0$`:

`$$\hat{\beta}^{\text{ridge}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda\|\beta\|_2^2$$`

- Solution:

`$$\hat{\beta}^{\text{ridge}}=(X^tX+\lambda I)^{-1}X^tY$$`

- .stress[K-fold cross validation]:

- `For each` `$\lambda\in\{\lambda_1, \lambda_2, ...,\lambda_J\}$`

]

`$$\text{CVE}(\lambda)=\frac{1}{K}\sum_{k=1}^K\text{error}_k$$`
]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Ridge regression ( `$\ell_2$`-norm ) <h0br>
.pull-left[
- In general, for any `$\lambda>0$`:

`$$\hat{\beta}^{\text{ridge}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda\|\beta\|_2^2$$`

- Solution:

`$$\hat{\beta}^{\text{ridge}}=(X^tX+\lambda I)^{-1}X^tY$$`

- .stress[K-fold cross validation]:

- `For each` `$\lambda\in\{\lambda_1, \lambda_2, ...,\lambda_J\}$`

]

`$$\text{CVE}(\lambda)=\frac{1}{K}\sum_{k=1}^K\text{error}_k$$`

]

---
count:false
## II. Hands-on Linear Regression Models<h0br> 
### &nbsp; Ridge regression ( `$\ell_2$`-norm ) <h0br>
.pull-left[
- Now, look!

|Model |  RMSE| Inter| total_bill|  size|
|:-----|-----:|-----:|----------:|-----:|
|1     | 1.039| 1.127|      0.093|    NA|
|2     | 1.070| 0.817|      0.077| 0.249|
|3     | 1.092| 0.853|      0.080| 0.228|
|4     | 1.033| 1.089|      0.095|    NA|
]
.pull-right[
<img src="itc_linear_files/figure-html/fit4-1.png" style="display: block; margin: auto;" />
]

### &nbsp; .subtitle[Other regularized linear models:] <h0br>

- Lasso regression: `$\hat{\beta}^{\text{lasso}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda\|\beta\|_1.$`
  
  - Elastic net: `$\hat{\beta}^{\text{elas}}=\arg\min_{\beta\in\mathbb{R}^{d+1}}\|Y-X\beta\|^2+\lambda_1\|\beta\|_1+\lambda_2\|\beta\|_2.$`

---
template: inter-slide
class: left, middle
count: false

<br>

<br>

<br>

<br>

---
## III. Logistic Regression<h0br> 
### &nbsp; Perceptron: 1st step towards Neural Networks <h0br>

- Start with simple case: .stress[Binary classification].

- Assumptions:

- Data : `$(X,Y)\in\mathbb{R}^d\times\{0, 1\}$`.
  
  - Sigmoid function : `$\sigma(t)=1/(1+e^{-t})$`.
  
  - Model : `$\mathbb{P}(Y=1|X=x)=\sigma(x^t\beta)$` for some `$\beta\in\mathbb{R}^{d+1}$` and `$x\in\{1\}\times\mathbb{R}^d$`.
  
  - Prediction : `$\hat{y}=1\Leftrightarrow \mathbb{P}(Y=1|X=x)\geq 0.5$`

]

]

---
count:false
## III. Logistic Regression<h0br>  
### &nbsp; Perceptron: 1st step towards Neural Networks <h0br>

- Start with simple case: .stress[Binary classification].

- Assumptions:

]

]

- What are the keys?

- .stress[YES! TO ESTIMATE THE BEST PARAMETER] `$\beta$`.

---
## III. Logistic Regression<h0br> 
### &nbsp; Loss function <h0br>

- .stress[Log conditional likelihood:]

`$$L(\beta|{\cal D}_{\text{train}})=\sum_{i=1}^{n_{\text{train}}}\left[y_i\log\left(\sigma(x_i^t\beta)\right)+(1-y_i)\log\left(1-\sigma(x_i^t\beta)\right)\right]$$`
--

- .stress[Log loss or cross-entropy:]

`$${\cal L}(\beta)=-\sum_{i=1}^{n_{\text{train}}}\left[y_i\log\left(\sigma(x_i^t\beta)\right)+(1-y_i)\log\left(1-\sigma(x_i^t\beta)\right)\right]$$`
--

- .stress[Challenge] : `${\cal L}$` is **smooth** and **convex** ([D. Jurafsky and J. H. Martin (2023)](https://web.stanford.edu/~jurafsky/slp3/)).

- .stress[Regularized loss :] for any `$\lambda=(\lambda_1,\lambda_2)\in\mathbb{R}_+^2$`,

`$${\cal L}_{\lambda}(\beta)=-\sum_{i=1}^{n_{\text{train}}}\left[y_i\log\left(\sigma(x_i^t\beta)\right)+(1-y_i)\log\left(1-\sigma(x_i^t\beta)\right)\right]+\lambda_1\|\beta\|_1+\lambda_2\|\beta\|_2^2$$`

---
## III. Logistic Regression<h0br> 
### &nbsp; Optimization in regularized case<h0br>

- Optimization formulation: given `$\lambda>0,$`
`$$\beta^*_\lambda=\arg\min_{\beta}{\cal L}_{\lambda}(\beta).$$`
--

- .red[CAN'T] be solved analytically!

- `$\tilde{\beta}\leftarrow\text{GDOptimization}(D,\lambda)$` : perform GD on a subsample `$D$` at fixed `$\lambda>0$`.

<h0br>

> - `for` `$\tilde{\lambda}\in\{\lambda_\min,...,\lambda_\max\}$`:
      - `for` `$k\in\{1,...,K\}$`:
        - `$\text{GDOptimization}(F_{-k}, \tilde{\lambda})\rightarrow\tilde{\beta}_\lambda$`
        - `Compute error on` `$F_k$`
      - `Compute CVE` for `$\tilde{\lambda}$`
- `$\lambda^*=\arg\min_{\tilde{\lambda}}\text{CVE}(\tilde{\lambda})$`
- `Use` `$\lambda^*\rightarrow\text{GDOptimization}(D_{\text{train}}, \lambda^*)\rightarrow \beta^*$`.

]

---
## III. Logistic Regression<h0br> 
### &nbsp; `Spam` dataset ([Hopkins et al. (1999)](https://archive.ics.uci.edu/dataset/94/spambase)) <h0br>
- `Spam` dataset : `$(x_i,y_i)\in\mathbb{R}^{58}\times\{\text{spam},\text{non-spam}\}$` for `$i=1,...,4601$`.

|   id| make| data| receive| address| num000| charHash| charDollar|type    |
|----:|----:|----:|-------:|-------:|------:|--------:|----------:|:-------|
|    1| 0.00| 0.00|    0.00|    0.64|   0.00|    0.000|      0.000|spam    |
|  100| 1.24| 0.00|    0.00|    0.41|   0.00|    0.000|      0.527|spam    |
| 1000| 0.45| 0.00|    0.00|    0.90|   0.00|    0.081|      0.162|spam    |
| 2000| 0.00| 0.46|    0.00|    0.00|   0.00|    0.000|      0.000|nonspam |
| 3000| 0.27| 0.00|    0.00|    0.00|   0.00|    0.000|      0.000|nonspam |
| 4000| 0.07| 0.00|    0.15|    0.00|   0.23|    0.000|      0.044|nonspam |
| 4061| 0.00| 0.00|    0.00|    0.00|   0.00|    0.000|      0.000|nonspam |

- Encode : **spam** `$=1$` & **nonspam** `$=0$`.

- .stress[Objective]: building `spam` filter using .stress[Logistic Regression].

---
## III. Logistic Regression<h0br> 
### &nbsp; `Spam` dataset ([Hopkins et al. (1999)](https://archive.ics.uci.edu/dataset/94/spambase)) <h0br>

.pull-left[
<img src="itc_linear_files/figure-html/inputs_spam0-1.png" width="100%" style="display: block; margin: auto auto auto 0;" /><img src="itc_linear_files/figure-html/inputs_spam0-2.png" width="100%" style="display: block; margin: auto auto auto 0;" />
]

.pull-right[
<img src="itc_linear_files/figure-html/inputs_spam1-1.png" width="100%" style="display: block; margin: auto auto auto 0;" />
]

---
## III. Logistic Regression<h0br> 
### &nbsp; `Spam` dataset : Results <h0br>
.pull-left-60[<h0br>
- **CVE** as a function on penalty parameters:

<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-193613dcf1b605329c22" style="width:432px;height:432px;"></div>
<script type="application/json" data-for="htmlwidget-193613dcf1b605329c22">{"x":{"visdat":{"ee855091c51":["function () ","plotlyVisDat"]},"cur_data":"ee855091c51","attrs":{"ee855091c51":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"inherit":true},"ee855091c51.1":{"x":1,"y":0.0001,"z":0.069007747142688897,"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"scatter3d","mode":"markers","showlegend":false,"marker":{"color":"red"},"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"camera":{"eye":{"x":2.5,"y":-0.69999999999999996,"z":0.69999999999999996}},"xaxis":{"title":"alphas"},"yaxis":{"title":"lambdas"},"zaxis":{"title":"error"}},"hovermode":"closest","showlegend":false,"legend":{"yanchor":"top","y":0.5}},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false},"data":[{"colorbar":{"title":"error","ticklen":2,"len":0.5,"lenmode":"fraction","y":1,"yanchor":"top"},"colorscale":[["0","rgba(68,1,84,1)"],["0.0416666666666667","rgba(70,19,97,1)"],["0.0833333333333333","rgba(72,32,111,1)"],["0.125","rgba(71,45,122,1)"],["0.166666666666667","rgba(68,58,128,1)"],["0.208333333333333","rgba(64,70,135,1)"],["0.25","rgba(60,82,138,1)"],["0.291666666666667","rgba(56,93,140,1)"],["0.333333333333333","rgba(49,104,142,1)"],["0.375","rgba(46,114,142,1)"],["0.416666666666667","rgba(42,123,142,1)"],["0.458333333333333","rgba(38,133,141,1)"],["0.5","rgba(37,144,140,1)"],["0.541666666666667","rgba(33,154,138,1)"],["0.583333333333333","rgba(39,164,133,1)"],["0.625","rgba(47,174,127,1)"],["0.666666666666667","rgba(53,183,121,1)"],["0.708333333333333","rgba(79,191,110,1)"],["0.75","rgba(98,199,98,1)"],["0.791666666666667","rgba(119,207,85,1)"],["0.833333333333333","rgba(147,214,70,1)"],["0.875","rgba(172,220,52,1)"],["0.916666666666667","rgba(199,225,42,1)"],["0.958333333333333","rgba(226,228,40,1)"],["1","rgba(253,231,37,1)"]],"showscale":true,"x":[0,0.010638297872340425,0.021276595744680851,0.031914893617021274,0.042553191489361701,0.053191489361702128,0.063829787234042548,0.074468085106382975,0.085106382978723402,0.095744680851063829,0.10638297872340426,0.11702127659574468,0.1276595744680851,0.13829787234042554,0.14893617021276595,0.15957446808510639,0.1702127659574468,0.18085106382978722,0.19148936170212766,0.20212765957446807,0.21276595744680851,0.22340425531914893,0.23404255319148937,0.24468085106382978,0.25531914893617019,0.26595744680851063,0.27659574468085107,0.28723404255319146,0.2978723404255319,0.30851063829787234,0.31914893617021278,0.32978723404255317,0.34042553191489361,0.35106382978723405,0.36170212765957444,0.37234042553191488,0.38297872340425532,0.39361702127659576,0.40425531914893614,0.41489361702127658,0.42553191489361702,0.43617021276595741,0.44680851063829785,0.45744680851063829,0.46808510638297873,0.47872340425531912,0.48936170212765956,0.5,0.51063829787234039,0.52127659574468088,0.53191489361702127,0.54255319148936165,0.55319148936170215,0.56382978723404253,0.57446808510638292,0.58510638297872342,0.5957446808510638,0.60638297872340419,0.61702127659574468,0.62765957446808507,0.63829787234042556,0.64893617021276595,0.65957446808510634,0.67021276595744683,0.68085106382978722,0.6914893617021276,0.7021276595744681,0.71276595744680848,0.72340425531914887,0.73404255319148937,0.74468085106382975,0.75531914893617025,0.76595744680851063,0.77659574468085102,0.78723404255319152,0.7978723404255319,0.80851063829787229,0.81914893617021278,0.82978723404255317,0.84042553191489355,0.85106382978723405,0.86170212765957444,0.87234042553191482,0.88297872340425532,0.8936170212765957,0.9042553191489362,0.91489361702127658,0.92553191489361697,0.93617021276595747,0.94680851063829785,0.95744680851063824,0.96808510638297873,0.97872340425531912,0.9893617021276595,1],"y":[0.0001,0.0051494949494949498,0.010198989898989898,0.015248484848484848,0.020297979797979798,0.025347474747474747,0.030396969696969697,0.03544646464646465,0.040495959595959599,0.045545454545454549,0.050594949494949498,0.055644444444444448,0.060693939393939397,0.06574343434343434,0.070792929292929296,0.075842424242424253,0.080891919191919195,0.085941414141414138,0.090990909090909095,0.096040404040404051,0.10108989898989899,0.10613939393939394,0.11118888888888889,0.11623838383838385,0.12128787878787879,0.12633737373737372,0.13138686868686866,0.13643636363636363,0.14148585858585858,0.14653535353535352,0.15158484848484849,0.15663434343434343,0.16168383838383837,0.16673333333333332,0.17178282828282826,0.17683232323232323,0.18188181818181817,0.18693131313131311,0.19198080808080809,0.19703030303030303,0.20207979797979797,0.20712929292929291,0.21217878787878786,0.21722828282828283,0.22227777777777777,0.22732727272727271,0.23237676767676768,0.23742626262626262,0.24247575757575757,0.24752525252525251,0.25257474747474745,0.25762424242424242,0.26267373737373734,0.26772323232323231,0.27277272727272728,0.27782222222222219,0.28287171717171716,0.28792121212121213,0.29297070707070705,0.29802020202020202,0.30306969696969699,0.3081191919191919,0.31316868686868687,0.31821818181818179,0.32326767676767676,0.32831717171717173,0.33336666666666664,0.33841616161616161,0.34346565656565653,0.3485151515151515,0.35356464646464647,0.35861414141414139,0.36366363636363636,0.36871313131313133,0.37376262626262624,0.37881212121212121,0.38386161616161618,0.3889111111111111,0.39396060606060607,0.39901010101010098,0.40405959595959595,0.40910909090909092,0.41415858585858584,0.41920808080808081,0.42425757575757572,0.42930707070707069,0.43435656565656566,0.43940606060606058,0.44445555555555555,0.44950505050505052,0.45455454545454543,0.4596040404040404,0.46465353535353537,0.46970303030303029,0.47475252525252526,0.47980202020202017,0.48485151515151514,0.48990101010101011,0.49495050505050503,0.5],"z":[[0.085849681866383906,0.077426641691781303,0.075241646223475595,0.074439684399509501,0.073083202021748894,0.074707009020688497,0.074429390566562906,0.073627416703026194,0.073352763996482495,0.073075833804481904,0.0744337930362481,0.073337242983257903,0.0738829545844919,0.073620056512139501,0.072534564804835305,0.071709749869089706,0.073076469895133694,0.071451926475786506,0.074440412793536706,0.071718619019503799,0.071173651865057905,0.072257658691976104,0.072815040650406407,0.072530886715987203,0.072525739799513794,0.0728033582536394,0.072812223390863198,0.070635236243666702,0.073078795538873503,0.072804851160405901,0.072803342200878501,0.072274644519635597,0.072537410156710197,0.073074352937285994,0.072540464194477197,0.073905051209912601,0.071999931615238405,0.072809149287145103,0.073076574238079797,0.070088916644112501,0.069817917947275696,0.073078783499302899,0.072798927691622606,0.069541003808036003,0.073624366678449404,0.073617102804128406,0.072264322594357305,0.0717178866122863,0.072802621833231704,0.073073616516878395,0.072794497129605798,0.071995509079602199,0.072527836691410399,0.071452018779161894,0.073609722547290604,0.073072787793095298,0.072273151612869097,0.074711447609085796,0.071178714504536403,0.073623630258041806,0.070907820150645604,0.071709765921850604,0.071719375505862604,0.072267143867090905,0.072527941034356405,0.073076586277650499,0.071722929185813605,0.070888657167284902,0.070635236243666702,0.073083958508107796,0.072807028316107095,0.071433600242589296,0.071721568714324793,0.072276097294499797,0.070904134035416802,0.072806203605514305,0.073623762693319397,0.073623726574607404,0.073622994167389794,0.070653807281468195,0.069543989621569005,0.072264286475645201,0.072001424522005003,0.071448344703503905,0.072539619417933299,0.070365102389325002,0.072267268275988103,0.071993311857949799,0.070091757982797195,0.071448244373748096,0.070364337876585706,0.071442453340242504,0.071998478840374205,0.072534588883976805,0.069007747142688897],[0.085849681866383906,0.079870829051259404,0.079859750639622307,0.079330256340118296,0.079601271089716,0.079327318684868003,0.079321435347986996,0.081235379947982606,0.082050645497569796,0.080949640803421402,0.078780886715987195,0.079051157018796794,0.079043796827910295,0.079870804972118001,0.080142536076172305,0.079855300011654307,0.080409007894426704,0.080414899257688202,0.083141264055396097,0.080957749454286396,0.080957021060259093,0.082034388063937494,0.079875839519264802,0.079596132199623096,0.083664110505279901,0.084203146164561996,0.081234683659477294,0.081496700836573593,0.083406379415352006,0.080959994834221602,0.082582296886823794,0.0804187017554336,0.082044597619889206,0.080951097591475896,0.080691937805825203,0.079878905596602504,0.080149904293439406,0.081230116648992601,0.0820432331352101,0.083398991132133796,0.080141075274927601,0.082312030597204502,0.080949648829801796,0.079315503852823196,0.082859815011193494,0.082312034610394699,0.082595592586064401,0.083667804646889096,0.082041772333965493,0.082853287557280303,0.083117650444067506,0.080955532166682803,0.083669141039236505,0.083676617612639795,0.083664818833355997,0.082308236125839498,0.082048392091254099,0.082315740791574402,0.081234535171438596,0.081496700836573593,0.082314976278835203,0.080949636790231094,0.080685306008965701,0.083676505243313298,0.082311298189986906,0.082859198986492896,0.081227907387769499,0.083387208405610896,0.081770649228231407,0.082859215039253906,0.082582288860443401,0.082308364547927004,0.082047687776368394,0.083958710773874706,0.083396041437312801,0.080684429126900006,0.080141827748096195,0.083952046871493505,0.083128716816133902,0.083694347887087406,0.082316469185601801,0.084216397718710007,0.083136853559330495,0.085303394372351501,0.083675768822905602,0.082321652220787106,0.083945411061443806,0.083407848242977201,0.083943825851302001,0.083671350300459593,0.083401192366976407,0.086660617183710098,0.0842179106914277,0.0825808240460085,0.081775067750677596],[0.085849681866383906,0.081499786979862501,0.083121368664817993,0.082861396208145296,0.082043973568808104,0.083672939523791595,0.083125791200454394,0.083404134035416896,0.084225302987836106,0.083941724946215199,0.084212731669432606,0.086385920332073804,0.084204635058138202,0.0844903782158897,0.083675148785014697,0.0836559657357029,0.084756833981383298,0.085575733474725996,0.084772443284793095,0.085576586277650496,0.084491110623107199,0.0844793038174428,0.086394632968068805,0.084758443270666603,0.086379296561594904,0.085288629845485797,0.086392576208074598,0.085572787793095295,0.085578823631205198,0.085036081790743304,0.086111215454057199,0.085035321291194205,0.085032255213856503,0.086385888226551993,0.085580292458830295,0.086393288549340905,0.086130398503368996,0.086119211735595697,0.086114901569285807,0.087474349694628306,0.086936790889351898,0.087206296679422105,0.087195978767334203,0.084746608372670701,0.088566336750324007,0.088560573809149903,0.0877571792762709,0.086656943108052095,0.089925921324134503,0.089373573912850807,0.088012652946692901,0.0863888539741339,0.089647401908801702,0.086662798352601503,0.088286609364731206,0.088556763285024195,0.088567942026417004,0.087204103470960095,0.088025804171085201,0.089916195357605797,0.088838940723253906,0.090458317160176702,0.089109239118395203,0.090196771532932707,0.090191012604948995,0.089646793910481498,0.090461882879698405,0.090178485431637895,0.089104660068339794,0.089382443063264899,0.090462735682622794,0.091273554617432306,0.089922243235286303,0.090746297671482803,0.090458321173366898,0.091008306822198598,0.091822211900296893,0.090200578043868399,0.091276492272682405,0.091839149569673706,0.089921490762117598,0.092638113540535999,0.091830312524781493,0.090739661861433202,0.093455431836927103,0.092370845104020299,0.092365650029264196,0.091830296472020595,0.091006833981383303,0.092092317662307105,0.093993867524269303,0.095351122441149697,0.092096876646411294,0.0915438048542907,0.093453958996111794],[0.085849681866383906,0.083129489355253702,0.085564075157100405,0.083949089150292103,0.084216413771470905,0.085849814301661498,0.085840236823171406,0.085577310658487504,0.086401425292537498,0.086114169162068405,0.084483001972242205,0.086115654042454401,0.086648818404426203,0.086118603737275395,0.086931595814595794,0.085560356936349904,0.086929274184046196,0.086388741604807404,0.088029630747972001,0.088020765610748106,0.088566465172411499,0.086381481743676505,0.088567809591139399,0.087746841298231695,0.087466253083334097,0.086649542785263198,0.087749799019433097,0.087745964416165903,0.088022266543895195,0.088841914497216304,0.088555394787154795,0.086936762797020201,0.088291651938258497,0.089644548530546306,0.089112905167672701,0.088296947342770396,0.089390535661368897,0.089650347590432403,0.089917776554557294,0.091546018128703999,0.091013618279471506,0.090739649821862597,0.090997384924980704,0.089366177603252203,0.0926409508660304,0.091549712270313097,0.091560794695140404,0.091278737652617598,0.094000539453031107,0.093447451608149601,0.092631497796437506,0.0920902207704105,0.094809708966654901,0.093996821232280606,0.093177181305339904,0.094262548603746901,0.093186046442563703,0.093996124943775197,0.093460586779780905,0.095620507835513097,0.094272990924732195,0.095346687865942803,0.096167824712840203,0.095899611170024796,0.095083805846351302,0.096436598095692994,0.096166195357605705,0.096155281486768204,0.096713355720513802,0.096174452496509397,0.097253280301432293,0.097250346659372294,0.099427953844459696,0.098355001350037205,0.101327174049492,0.10024154589372,0.101875094911949,0.098622293865694197,0.102146826016003,0.101076823216402,0.099698212107698594,0.10105982936236201,0.10378169136062899,0.10024538852336801,0.101603187227524,0.10269768854690101,0.10323154518418801,0.100792540859895,0.103232149169318,0.102416195357606,0.10377281017064401,0.10295614401766599,0.10432073103310099,0.103498129371769,0.10350462472016],[0.086120684576411005,0.084486716179802496,0.087193773519301396,0.085307048382058298,0.086935281929824401,0.086936770823400705,0.087739465054584201,0.0869352698902537,0.088031123654738502,0.087472864814242296,0.088014137827078995,0.087745352404655405,0.089094470578339205,0.088020777650318904,0.088561294176796798,0.085833576933980402,0.088832920937905097,0.088290915517850801,0.091563720310819899,0.089651200393356806,0.088567201592819195,0.089096663786801394,0.089926505243313401,0.089375803240025004,0.089637212418801099,0.090452417770534796,0.090464981062558,0.089920613880051903,0.091554142832329904,0.091013618279471506,0.091543056394312303,0.090470848346677998,0.091824260633910706,0.095078606758405002,0.093461479714607595,0.092644781456107397,0.092105717704493703,0.093453222575704001,0.093448184015367003,0.095348152680377704,0.0948142840035201,0.094541788386726403,0.096156754327583499,0.095341512857137795,0.096716301402144503,0.095896805950052094,0.097535377475857402,0.095896838055574002,0.098074421161520001,0.097791591579877393,0.099969198764964795,0.098877803654828303,0.10024154589372,0.098620829051259407,0.10051414986665,0.098882850241545903,0.098618623803226502,0.102958361305269,0.101339548721574,0.10105381760339301,0.102423700023341,0.102679233891616,0.10295836531845901,0.103233622010133,0.102412661743606,0.10404457739341,0.10431689642983399,0.102130608714273,0.10486258395192601,0.103780946913841,0.10676121622459001,0.10567502020239999,0.10703297542097601,0.107864402087629,0.107306927825824,0.107305290444209,0.108941068789612,0.106769320862264,0.10649466012934,0.106779666866684,0.110562630408617,0.11083802756194799,0.108128793066748,0.108941060763232,0.11084084482149199,0.110840284981454,0.108665591372476,0.10785776226439001,0.11111037469070301,0.111109638270296,0.11192351926925299,0.110835854419437,0.110841001335911,0.111372676804306,0.114099505125486],[0.087478643808177201,0.0850287175866664,0.087468470370937607,0.086937491191047406,0.086663542799389606,0.088020041229911097,0.089366958168752397,0.088294705976025498,0.089389082886504698,0.0891025631764433,0.088012664986263603,0.089376527620861998,0.090453906664111003,0.089380213736090702,0.089648254711726105,0.089094454525578307,0.091280045952633504,0.090192353010486698,0.093465157803455795,0.093453350997791507,0.091557076474389903,0.090724889308187007,0.091828679156356702,0.094265642773416203,0.092352398475116101,0.093170545495290205,0.092909164408845904,0.093453222575704001,0.095901236512068902,0.097259944203813606,0.095885735564795502,0.094816473198791898,0.096171350300459493,0.098607529338828506,0.097262885872254096,0.097807828947558503,0.096453555831020998,0.096983622010133094,0.097525023445057299,0.10023577492616501,0.097528737652617603,0.098617146949220996,0.10131906539862701,0.098600173161132093,0.101064127489101,0.100240953948161,0.10242742225728101,0.10105914511342701,0.103512901925015,0.10268069469286099,0.103772081776617,0.104582864592714,0.10458716271945299,0.104325145542357,0.107030734054231,0.104588635560269,0.106493167222574,0.10622366945888401,0.105414899257688,0.106488600212089,0.106773015003873,0.10648358974408401,0.108391687125911,0.10703723341581201,0.10730471455141,0.10893146321879001,0.10839151054554,0.10620522684317001,0.109750942618122,0.110570759125433,0.113009747396965,0.110566328563416,0.113010511909704,0.11166802151968901,0.112739505186487,0.110837899139861,0.111659188487987,0.112748322165428,0.115191060763232,0.113843423456744,0.114095243117459,0.117090252875932,0.11274911075730799,0.11709251832181899,0.117361847531519,0.116269920673677,0.11898800826653,0.116283900621852,0.117903116531165,0.118174119241192,0.11817353130882299,0.118986563518046,0.117085850406247,0.118441608403171,0.120616825733475],[0.089108342170378205,0.086656943108052206,0.088282951341834295,0.088022978885161404,0.086661329524976294,0.089653437746911493,0.090453178270083798,0.089380930090547303,0.090476039408243905,0.091275003379106101,0.089369887797622199,0.091276488259492305,0.091268383621817606,0.091280178387911096,0.092093170465231494,0.091538633858676,0.092638741604807298,0.092910480735242107,0.095096329006472094,0.096441012604949,0.095632427010504006,0.094258978871035001,0.094545334040296899,0.097525779931416201,0.094795849414186603,0.095887196366040206,0.0953533437419435,0.096169141039236503,0.098617895409199297,0.099431651999259102,0.099148094023589303,0.098078087210797499,0.100247437256981,0.10024091381625801,0.100520809755841,0.102970144031792,0.10242813459854801,0.101058972546247,0.103768399674578,0.101866946129182,0.103233790564123,0.10486420126759,0.10511973914905599,0.10375879811694699,0.107583657358313,0.106487271846122,0.107316517343884,0.105677981936791,0.108947692560091,0.105939354996855,0.107305434919057,0.109473448572893,0.10947699422646399,0.108941040697281,0.111107565457541,0.108932779545187,0.11192796187084,0.110837347326204,0.110034464475079,0.111647961588311,0.11220559637772699,0.113276347637306,0.11491196946829101,0.11192780134323101,0.11138154996791,0.11545543568959001,0.11491251325556701,0.113544408678893,0.11382555673382801,0.115190328356014,0.11490749877437199,0.11681559615619901,0.118984358270013,0.117103552588363,0.11817282298074699,0.116812477907388,0.118721460197661,0.118453379090123,0.118996157049297,0.120633960049173,0.11952857295129,0.121161921310008,0.12062808072548201,0.12116492317630199,0.12306542358901899,0.12251771943883399,0.12550533288770899,0.122529478086216,0.12442338282078499,0.12469217626958901,0.12414885452313799,0.12468794235389299,0.124963347533606,0.12632057034496399,0.12659435018263199],[0.089923559561682506,0.087472900932954303,0.089369175456355907,0.089380942130117894,0.087477287349878599,0.091012133399085399,0.090453178270083798,0.091012845740351705,0.092920963188129702,0.092362696321252893,0.091269848436252396,0.093990933882209401,0.093441560244888006,0.092639610460492697,0.096166311740122495,0.094254552322208501,0.0948104453870625,0.095626399198774595,0.095910809977368797,0.098340973243579197,0.097533864503139694,0.096975641781355607,0.097532991634264196,0.098612000032747602,0.096698767774018202,0.098876334827203302,0.097798263508639197,0.098886532343584396,0.10051786006102,0.10214757046279099,0.101593009777095,0.10215270533969401,0.10241914103923599,0.10458801150918801,0.104325165608308,0.104605017402798,0.106773019017064,0.106764021444562,0.10892924191799699,0.105398822417616,0.107307676285802,0.10649241876259501,0.108651619450681,0.107562421562197,0.111656798633204,0.111103895395073,0.110849138079107,0.11111129973105199,0.11275130396577,0.109743714862513,0.11274170240813899,0.112735799005307,0.114367562153882,0.113830135783884,0.11599888184493699,0.113007393660893,0.11654679869420401,0.114914919163112,0.114381554141628,0.116537793095322,0.117098393632319,0.116807483492143,0.118712635192339,0.11790458937198101,0.115727911240432,0.119259051108459,0.119259602922116,0.11789150235863199,0.118987863791681,0.120896129727498,0.121154561119121,0.122250394817655,0.123332932816948,0.12389336078676499,0.122520657094084,0.123332744197007,0.123064127328573,0.12578739795661201,0.126873654176656,0.12552379155618401,0.126864805092193,0.12877432314336201,0.126060658086145,0.12822498963794299,0.131759602922116,0.129582948869709,0.13202560720370901,0.13094971304084599,0.131761075762932,0.132569665370567,0.133390242377427,0.13147922339311099,0.13420175358755099,0.13637126014815401,0.135558059384942],[0.090467037822552193,0.088559121034285898,0.090453922716871998,0.090739641795481996,0.08856350745121,0.092912094037715595,0.091810401081442394,0.091828063131655993,0.094007187302651299,0.093448916422584502,0.093714027769350103,0.0937184583313668,0.094530734054230695,0.093996096851443597,0.096437314450149594,0.096155253394436493,0.097256097560975599,0.096714092140921401,0.096452815397422995,0.100242414749405,0.099707781559808195,0.099964780242518606,0.101064863909509,0.100513437525383,0.098330683423822798,0.100777772319839,0.102417096318813,0.102147401908802,0.103233046117335,0.105135224043568,0.10512783977354,0.105956324771754,0.105136532343584,0.106489449001823,0.106499086678167,0.10867963954488501,0.110035361423097,0.110025627430187,0.11191837636597,0.10974370683613199,0.111927997989552,0.111113472873563,0.112182759318708,0.111910255675534,0.114916195357606,0.114637248537513,0.114923756208004,0.112742470934068,0.114924484602031,0.114903076238735,0.11573010444889401,0.11572345659927399,0.11789796158831201,0.11573230969692699,0.11790252859879601,0.117354483327442,0.120349677692666,0.118175056321112,0.11899964651820399,0.11979645339931699,0.12035852677712899,0.120881365200632,0.12170250606072,0.121163986096383,0.12414888662866,0.123334417697334,0.122790002356545,0.12386903483417,0.12469291268999599,0.12578670568129699,0.12822053499678501,0.12985837812856299,0.130937942353893,0.12905640827821599,0.129853939540165,0.13148270884882801,0.13311849522061101,0.13257646572141599,0.13420841347674201,0.132584590425042,0.13365681853862699,0.13556560819576899,0.13339320411181799,0.13474304666633899,0.1363784317191,0.135288005794405,0.13772772245996401,0.13937512300428101,0.13746686108165401,0.138002975138447,0.13719459421670399,0.136914006001806,0.140722772350339,0.140717613394295,0.13990514905149101],[0.092640214445622607,0.090462039394117497,0.091810409107822899,0.092097601027248302,0.090467162231449294,0.093455572298585296,0.093710365733262899,0.093457025073449398,0.094549925129922996,0.095077878364377796,0.094802465158285096,0.095619159403595,0.096160432416431699,0.095626535647242494,0.097797486956329102,0.099145128276007299,0.097256097560975599,0.099431483445269198,0.100529650813923,0.101871372678008,0.101065004371167,0.10267996228564299,0.103509779663014,0.10349814943772,0.101864776999861,0.10349812134538799,0.10377800925859,0.106495227995758,0.10513522003037799,0.108934412913611,0.108114024526692,0.109759203770216,0.109485094850948,0.110291591579877,0.110575918081477,0.111122346037167,0.11221075533377101,0.113829975256274,0.11218937907599701,0.113006057268546,0.11464612972749801,0.113559865481074,0.115171897779871,0.11517186567434901,0.11790458937198101,0.116540162884155,0.11764189195914,0.116546826786536,0.120629545539917,0.11707551644139801,0.11953371585457399,0.11871333148084499,0.119799399080947,0.119806195418606,0.121161925323198,0.12007187463179,0.123883767255514,0.123880845653025,0.122530045952634,0.12713120065983299,0.12578889086337899,0.12549651590876801,0.125506861913187,0.12877194532814901,0.12985319910656801,0.13148513080913299,0.12903779309532201,0.13011682557294699,0.13039869800871901,0.134480156620367,0.13446906616915999,0.136923607559438,0.13419882395868099,0.13421651410122701,0.13610247873892101,0.13990662189230599,0.13773659963675799,0.139634318908643,0.139371460968193,0.13720490410241101,0.13691842853744299,0.14100409698564501,0.14072796341190499,0.140722064022263,0.14262622245787701,0.14370824877541499,0.14424947362996901,0.14453891895571,0.14398933663249699,0.142078325674561,0.14262865244456299,0.143434280317806,0.14615387687018699,0.14642193389858299,0.148328325674561],[0.093454695416519407,0.092091737756318404,0.092625630512317397,0.092096124173242602,0.091554122766378795,0.094542532833514506,0.095069065398627001,0.095357726145677502,0.095093407403982894,0.096163362045301501,0.096703906664111106,0.097250338632991803,0.097519132081795898,0.096712015314976099,0.099153973347279906,0.10104730620224101,0.099157535053611398,0.101875662778367,0.10161439807443901,0.102959069633345,0.103509183704265,0.104310401081442,0.10812787203959,0.10648728388569299,0.105668400445111,0.107842997737524,0.106764198024933,0.109756833981383,0.10920762488486201,0.11165180823114999,0.111645900815127,0.11247512624693801,0.11247496170613901,0.113553946025481,0.11302157025539,0.114653489918385,0.11492889509809701,0.116002415458937,0.11544878382677901,0.11681041312101299,0.117089572640188,0.114918565146438,0.118974036344735,0.117616789454235,0.120077765995051,0.117627119405894,0.120089025000249,0.120077966654563,0.12361868400108,0.122235626277599,0.12360981485066599,0.123058956332959,0.124689967008366,0.12524319531490499,0.12822568993963801,0.12713340992105601,0.127957661003574,0.127685193479112,0.129863320372334,0.13093407564510401,0.13258090430981301,0.12984360557531599,0.13365758706455699,0.13393425238600201,0.13365828736625199,0.139634359040546,0.135558059384942,0.135280613497996,0.13773491811005101,0.13855551518286199,0.13963286212058901,0.14236061548211701,0.14072131556228501,0.139376615911047,0.14289301533134999,0.14533845881937099,0.14072573408473099,0.14343941920789899,0.144806984010487,0.145630310052656,0.14262422589573601,0.145348985417351,0.14561706251169801,0.14805608288875199,0.14914722516790399,0.14968504483054501,0.149684272291426,0.146985311563221,0.148063950748203,0.149414545775893,0.14969166860102401,0.15130735089633801,0.14941105630698501,0.15212699082327899,0.15185725226817501],[0.0939974372569813,0.093177965884030406,0.093710377772833503,0.093183817115389506,0.092640342867710196,0.095901968919286401,0.096698763760828005,0.096445423101014602,0.096723842186591497,0.09752058485666,0.097516914794192305,0.098880041008383102,0.098604611749529406,0.098069974546742295,0.100785144550296,0.103492966402535,0.101058972546247,0.10404810298103,0.10569196991134699,0.10458950441595399,0.106499799019433,0.107028544858959,0.108941616590079,0.108930726798383,0.10675535696685,0.11110535619631801,0.10866342625634499,0.112472016024508,0.111651067797552,0.11545763692443201,0.113819821884986,0.11437804460676999,0.11356191822787801,0.11626839164819799,0.114923007748026,0.116828147408651,0.116558597473488,0.11899228231412801,0.118979187274398,0.12007055027901301,0.120079443508568,0.118179442738036,0.12223270066192,0.12087619019182699,0.12524080947331201,0.12224669264966501,0.124432432564759,0.12333810381256299,0.128236039957248,0.12685447112734399,0.129585882511769,0.12767778914313299,0.130394279486273,0.132032255053329,0.13230030806853499,0.131207287616354,0.13338949793063901,0.134475001677513,0.13529589371980699,0.13799855661600099,0.136383791334655,0.135547922066414,0.13882064258238799,0.13855234476257799,0.13963434298778499,0.14316549489538299,0.14207906209496901,0.141531361957975,0.142081271356192,0.142627916024155,0.144250218076757,0.14616201762657399,0.146425636066573,0.14399397186721499,0.14914229496370299,0.148055850123719,0.14561925973335099,0.149413269581399,0.15186778689253499,0.15106215500610201,0.14995897315625201,0.15322427328348601,0.149692417061003,0.154028456408246,0.153766790385295,0.15403361536429,0.1543031051016,0.152417896950264,0.15404073877695301,0.153762371862849,0.155396717499339,0.155383453905621,0.15457337139121899,0.15810304644481099,0.15620434193472399],[0.094269176387416095,0.093992446854926998,0.095340076135034604,0.094541039926748005,0.093997565679068695,0.097262145438656203,0.098055986572186601,0.097804118753188493,0.097537586737080503,0.099694501913328598,0.099418352286828202,0.101055434919057,0.10023357369132301,0.10160259126877499,0.101056883680731,0.10512340519833301,0.103232149169318,0.10622054318369301,0.108137630111641,0.107576429602704,0.10921498507574801,0.108930722785193,0.110842317662307,0.11110464786824099,0.109743014560817,0.11354806670179,0.111650347429905,0.113830711676682,0.114908991681138,0.116271381474921,0.116809696766557,0.11709322664989499,0.11600683398138301,0.116812610342665,0.11954184858458,0.119816541423026,0.118462256266918,0.12170820077765999,0.121968321722371,0.12332773372900201,0.123881590099813,0.123613496952704,0.125761635281914,0.12549281775396801,0.13013211382113801,0.127951025193524,0.127148347015101,0.13012495830295301,0.13176939711287899,0.13092909326943,0.135021409567253,0.13229146299726199,0.13555585012371901,0.136651091876693,0.13583218435697,0.138810091905267,0.136108378128563,0.14045622428147,0.13964224696594801,0.14397534464475101,0.14344532662392101,0.14233993952603899,0.145341653318796,0.14398639095086599,0.14289817830058399,0.14724306673229001,0.14669715447154499,0.14669368105539801,0.14724431483445299,0.14968650563179001,0.151312493799621,0.149150411640949,0.15212994854448,0.15024177464556299,0.15267564810614301,0.15213267350064799,0.151867798932106,0.154032106404763,0.157574316658046,0.157309953771259,0.15647850703865401,0.15811336837008899,0.155397473985698,0.15918855821806599,0.156211706138801,0.158379972623621,0.16028359529833899,0.15893891571305199,0.161644279486273,0.16326882290562,0.15974602846668201,0.16163124464439799,0.16299361838542001,0.162989199862974,0.16543905384706001],[0.095355396488747496,0.094535929128986798,0.095883554395904097,0.095085254608025305,0.095354788490427195,0.097533884569090998,0.099142206673518002,0.099433817115389497,0.099710026939743401,0.102138681246426,0.1007777883726,0.101056907759873,0.102133542356333,0.10350476919500901,0.104043072447073,0.107841540949469,0.10676475786497,0.108122717096736,0.10976806489424901,0.108935129268068,0.111660641262851,0.111107589536682,0.111929274184046,0.11300756221488301,0.112731408575192,0.11707994299022501,0.115724965558801,0.116276363850595,0.117084385591812,0.11817281896755701,0.121427052722725,0.12035336782108499,0.11953944267703499,0.119529269239796,0.122801245308982,0.123348417711461,0.121722389411727,0.126054554023801,0.12549946560358899,0.12821535998798,0.12741272595465,0.127687374648003,0.13010872896165299,0.129293483478017,0.133390774125133,0.131754640612394,0.13230991765254699,0.13501478980996401,0.13421726657439501,0.13527324126753901,0.13991272595465001,0.13799578350155001,0.13963193708023999,0.141540931410084,0.14045028475992599,0.141798485919642,0.14208442973690499,0.14426057210755699,0.143174119241192,0.14804848591964201,0.14724746920197601,0.14831821644836499,0.15186414090920899,0.14996244255920799,0.148870551820078,0.15240612225012201,0.152401466949452,0.150767554737507,0.15077618710969701,0.15484660744161,0.154843637680838,0.156212687363812,0.155934304396948,0.15404613049803101,0.15648220920664399,0.15620802403676201,0.15703306772435099,0.15919441747580601,0.162462679337432,0.160570090929259,0.15919958445823101,0.16245824877541501,0.159745304085845,0.16489730527118099,0.16055879580534901,0.16517124563645799,0.165722824521812,0.16491423892736701,0.16789207022505001,0.16761370331094599,0.16545257027176399,0.168692787960044,0.170598635948745,0.170324699596658,0.17087089077412501],[0.095627135619182194,0.095622885650726006,0.095884294829502004,0.096988913401454796,0.096441749025356599,0.098077362829960504,0.101587858847431,0.100792516780754,0.10161146443237901,0.102953902650921,0.1040379255306,0.102686602108883,0.104576985269023,0.105135212003998,0.10540176809924701,0.108385019210339,0.108938670908448,0.10975315187934501,0.11139776325645,0.110564831643459,0.11329108005865,0.113009027029318,0.115461882879698,0.11626843980648099,0.116262548443219,0.119256077334497,0.116811922080541,0.117906062212796,0.119530041778916,0.121160484587905,0.124686453460317,0.12334176183546,0.123615529633557,0.123603154961475,0.12524542464207999,0.12796429681362301,0.12661517863993901,0.130944385530812,0.129846559283328,0.130118274334621,0.129857641708155,0.13094677538559499,0.13282612427919099,0.13309488562247301,0.13719438553081201,0.136100993858535,0.13665258879664999,0.13745896914306199,0.14127955032364001,0.13989133364411499,0.144258350806764,0.14315662574496801,0.14397829032638201,0.14561924368059001,0.145072815724899,0.14560209732532101,0.146702522113481,0.14697723501787799,0.14833568987863799,0.15022239896312001,0.153226482544709,0.1518500927368,0.154851070109149,0.15376531754448,0.152672694398132,0.15539526071128501,0.15729056203605499,0.154840696012398,0.15566675503711599,0.158377747309637,0.158645780258893,0.162463443850171,0.160553145233502,0.160290987594747,0.16001187623385499,0.161371067515023,0.15974973464786199,0.163541519181926,0.16599454759948601,0.164914234914177,0.16354963987236101,0.16762203268727399,0.16572136773375801,0.17060310664266501,0.170334040296925,0.173050227644203,0.17115836361686701,0.169259123345883,0.17441380935548501,0.175219453281489,0.17360181054274701,0.17657102552100001,0.17711742939754899,0.17657249836181599,0.179565070107223],[0.096713355720513802,0.096166363911595498,0.097241517640860503,0.097531655241916607,0.097798975849905406,0.099707801625759401,0.102945818079197,0.101878736882085,0.102425945403276,0.10376912004222499,0.105666887472393,0.10540326100601401,0.106478422761659,0.106766391233395,0.108660428403242,0.108928497471209,0.11029810298103,0.111384323082361,0.113571676299928,0.114094494657481,0.116006262101775,0.115183684519584,0.117092317662307,0.11735466392100199,0.117892983225828,0.121429994391165,0.119801056528513,0.121709673618475,0.122250394817655,0.123063402947736,0.12767190179306101,0.12660263942705799,0.12687566277836701,0.12659155298904001,0.12742008213234601,0.13230991363935701,0.12960357265431399,0.13339003770472499,0.13283643817808899,0.13473564233036001,0.13257429659209499,0.13637714749822499,0.137988435350234,0.13635724809445701,0.14208348061741499,0.13854517720482301,0.141814899867693,0.14234806824285501,0.14616717658261799,0.14451238775508199,0.147517747531166,0.14641824377016399,0.14615220336986001,0.15078452049921501,0.15077861709638299,0.14967671144102701,0.15077862110957299,0.15105333401397,0.152682779545187,0.15565570873099999,0.15702936555636099,0.157829842499941,0.15756625215227399,0.15729718981972399,0.15566034797890901,0.157840912885198,0.16027821963002201,0.157827621199148,0.16028263815246799,0.16082340349674101,0.16163417828645801,0.16572506187536701,0.164356028245154,0.16545330267898101,0.16571914643296401,0.16707537999293001,0.16545479157255699,0.17033427707514801,0.17006990214879,0.16980407444756801,0.17088808930086699,0.17495973766899101,0.172238700381317,0.17685311868223499,0.17631082832567499,0.17766832804715901,0.177950377063302,0.178224325454959,0.18229056203605501,0.18282741251325599,0.184739444828105,0.18553622763007599,0.18580792662860901,0.18635291384900601,0.189615735831271],[0.097528573111818104,0.096709842172465005,0.098056735032164902,0.098619352197253707,0.098884459530829097,0.100795494567906,0.104032774600936,0.104051913505156,0.103512901925015,0.104311861882687,0.107569061385437,0.107576437629084,0.10892408296195299,0.10948378253774201,0.111375614459556,0.111370471556273,0.112199540473666,0.113014021444562,0.11628981205106401,0.114909712048785,0.118178702304438,0.117898866562709,0.119266230705785,0.119255364993231,0.11979368429805599,0.123602438607018,0.123061189673323,0.124425592082008,0.12442283502031801,0.12469236488953001,0.13038782426978399,0.128233818656455,0.12823435843054101,0.128764733625301,0.130954175708384,0.13529683481291599,0.13286370579912399,0.13583495345822999,0.136912529147801,0.13799651590876799,0.137193133415459,0.139367018366606,0.14043261468333201,0.14070360535378801,0.14643130670437099,0.14370601543505099,0.14507282375128,0.14588068095169801,0.14969978929145999,0.14831896089515301,0.15186484522409499,0.15076607387031099,0.149684075645104,0.15214322016457901,0.15430976097759999,0.15402380110757599,0.15349528803308399,0.15485768184005699,0.15621465182043101,0.15891731471662501,0.16083150010803501,0.16054650139707199,0.161369127137545,0.16164280664545799,0.15946249055696299,0.16327422666626901,0.16299561093437001,0.160545760963474,0.16381377400730501,0.16679946714465399,0.16760876508036501,0.170615629802785,0.167890113794812,0.172514101547839,0.16979081085385,0.17114852126782101,0.17143159164087801,0.17685086527592001,0.173875730842073,0.17822283254819299,0.17713441923839901,0.182295229376296,0.180385727377887,0.18174000852080599,0.18418831742665301,0.186907478547893,0.18718730626324201,0.18800547736255699,0.189895575586191,0.19043389890420601,0.19370168921597999,0.19423116344953301,0.196403555493913,0.19667755204385401,0.195049782019559],[0.098070578531872302,0.0972518475925193,0.098871216003061604,0.100519316849074,0.099699676922133607,0.102699149348146,0.104302300456958,0.105409140329704,0.103783900621852,0.105398081984019,0.108928497471209,0.108935873714856,0.110286476768926,0.11084174176950901,0.113277051952192,0.11354291175893599,0.11600167903853,0.116273418168964,0.117647030849233,0.118169108773187,0.120893884347563,0.120344518736622,0.121168404618829,0.120070582384535,0.122508133933963,0.12631983793774701,0.124145936933839,0.12768572522681801,0.128767723452024,0.12849523987480099,0.132289998182827,0.132583870057395,0.130680010604454,0.13121113024600201,0.13530127741450401,0.13801348969685601,0.13639558610074901,0.139366825733475,0.14153209837838199,0.14125591664636,0.14018153946940501,0.14507134288408399,0.14396523140536499,0.14532022890273899,0.14941822787793099,0.14778136998435501,0.148877187630128,0.15158426102238801,0.15051500266957399,0.15157614434514199,0.15593872291939401,0.15293925851976201,0.15294420878991399,0.15404392123680699,0.15784237368644299,0.156741192411924,0.155937994525366,0.16001705124266,0.15893057028396401,0.16190644515140801,0.16463363867289901,0.16435085724953899,0.16489731329756099,0.16463193708023999,0.16407763323871799,0.166260415432611,0.167886178861789,0.16652477029301699,0.17033477671733199,0.17060013286870199,0.173043555715441,0.17441703596043201,0.17576909178936601,0.177405409908856,0.17712556614074601,0.17848474136915299,0.17795186997006801,0.18119500926083801,0.17903730549270899,0.18637649936800299,0.18663940947992599,0.18990395312080099,0.18962118373701201,0.18935241035415901,0.19206801578885399,0.193696546312697,0.195603859115833,0.195340236662644,0.198318015788854,0.20075851301991299,0.20511401633978399,0.20537322829690799,0.20373757034721199,0.208084720211614,0.20809031459879801],[0.099157535053611398,0.098066328563415905,0.10077192108848,0.10106205868953599,0.10051489832662799,0.104057108579912,0.10484503828423,0.106766363141063,0.10568534212767799,0.106756041215785,0.11001471757254,0.111382270335557,0.11055895231976801,0.11084247818991599,0.114090796502681,0.115443612831164,0.11763137740073,0.11871833392247,0.120091214195521,0.119527068004953,0.12170762889805201,0.122247433083263,0.12279810298103,0.123602454659779,0.125223319990278,0.12957924268852899,0.12604737843966499,0.129316160009426,0.13257134288408401,0.13066842051106201,0.136364612298534,0.13665996905348801,0.13475536114056799,0.13501328085043701,0.138020149586047,0.14100188371123101,0.139384720548722,0.14398639095086599,0.145334977376844,0.144514576950354,0.14507138301598599,0.14724230623274101,0.148318236514316,0.14858036606073899,0.15322257570401801,0.15104003430154,0.14996266729786101,0.155659611558501,0.15377366698675901,0.15510580735916399,0.156752459443502,0.15511096230201701,0.15756303758689799,0.15567435601941601,0.16191699582853,0.160543330976788,0.16055682733554,0.163278669267856,0.16246170613880101,0.167068015788854,0.16843799452536601,0.16761247126154499,0.168428457178779,0.16734785554377299,0.168696478088463,0.17305244091861599,0.17223326852833701,0.17168634494365301,0.17603908919524,0.17521454314324,0.17902034775738099,0.17985256702910599,0.18310163781503899,0.18310973041314299,0.18120240155724601,0.18609196418051099,0.184740197301274,0.189889196620316,0.187729283590964,0.19425620976977501,0.19588003282147501,0.196967705697671,0.196141458053012,0.19994803119308299,0.20157814893366299,0.202119730962148,0.205651583171441,0.20593217941271899,0.208368681512902,0.21271356191822799,0.21543348353901701,0.21542832458297301,0.212971553865521,0.21650495516624399,0.220588105337575],[0.10078796983622,0.098338067693850797,0.101316135769757,0.101605536950406,0.101058376587498,0.10568754336252,0.107017478486892,0.10731058183552999,0.10704477821345,0.107843738171122,0.11055819583341001,0.113557664246231,0.11218865870835,0.11220043742168299,0.1159915015881,0.118431270425132,0.118990073052904,0.120892246965948,0.120363689746363,0.12142998235159499,0.12388080953431301,0.124149615022687,0.12578723341581199,0.12604959171407801,0.12766897216419101,0.131481416601573,0.12903577646723,0.13148933663249701,0.13365756298541501,0.135014781783584,0.13853557966038099,0.13910341196617701,0.13774301873453501,0.13854589355927899,0.14209772142295499,0.14426128043563299,0.142372382155879,0.14724652409567601,0.14805089985356701,0.14831597909481101,0.149418472682535,0.15131618794123,0.152121851933186,0.15265867030486399,0.157026923530105,0.15402843232910499,0.15213584392093199,0.15783352460198,0.15730406642118799,0.15728193769024501,0.16028359529833899,0.15837257631402299,0.16109490986214201,0.16110766980048699,0.164361911582035,0.16543389890420601,0.16463144947762701,0.16653807803182899,0.165720366442795,0.17032667609284799,0.17088143743805601,0.17223057166450101,0.17304951128974599,0.17414059738423501,0.17413274959073499,0.17821476001604,0.17847958642629899,0.17711670501671301,0.18147018970189699,0.182821045586952,0.18336671304309299,0.18610259110824701,0.19070887667915901,0.190173487003203,0.18989732533713199,0.19505420054200501,0.19370317810955601,0.19858117471857101,0.196150262992382,0.20186566190830699,0.20539462060744301,0.20538646781148501,0.204559483746419,0.207013260623958,0.21081580652762999,0.21353058524513699,0.217063922334816,0.216528548711621,0.22032078472958599,0.223580181453988,0.22440015447571801,0.224122515955642,0.22465266239855899,0.22846001610413,0.23281047484387901],[0.101874926357959,0.099695290505209297,0.101587874900192,0.1024207583549,0.102689555816895,0.106773027043444,0.10783195945778901,0.108398282804058,0.10785852276393899,0.10865821914201799,0.110286456702975,0.11437288565072599,0.113274878809681,0.114372141203938,0.117076248848616,0.12060518346861,0.120077029574644,0.121163986096383,0.121722389411727,0.123604635828671,0.12659746843144301,0.126864801079002,0.12904810298103001,0.12876846388562199,0.129300151393588,0.13311259181777901,0.13310965416252901,0.13447846706727901,0.135831476028894,0.13827418252117599,0.14125149812391399,0.14100411303840599,0.141547366560622,0.14126255245641001,0.144815120753684,0.14670399094110601,0.144273087241298,0.151866089313067,0.149137115941708,0.152935556351772,0.152134391146068,0.15321836185427401,0.15320954888852301,0.156189073752484,0.16110227406621899,0.15674509122623501,0.15784164529241601,0.15973570252821401,0.16192512454534599,0.16026738602299001,0.163542996035931,0.16081602323990299,0.16625721691999501,0.16518155150897601,0.167622052753225,0.16733607281724999,0.16843654576369199,0.17115543800118699,0.170339195239778,0.173858548368092,0.17740392904166,0.179020371836523,0.17984300560337699,0.18011812183339199,0.18038128878949,0.182832123998589,0.18337089077412499,0.183910195317153,0.18690276304937001,0.18934649892494701,0.190158726489528,0.19343440071351301,0.196410983909033,0.19587559423307799,0.19967847323154,0.202660686932956,0.20239294293339799,0.20755005892968501,0.20538498694429,0.208382986529486,0.211911945228622,0.21516245273665899,0.21433769398557601,0.21706541524158299,0.223041121715565,0.22358494912398499,0.22439868966128401,0.22548931223230001,0.23064318958407001,0.23363158359844499,0.23608421671676699,0.23825224241017401,0.23742442560813501,0.24177230386656401,0.24476552374219401],[0.102690143749264,0.100781510606541,0.10213135316106101,0.103777981166259,0.104318517758688,0.108131726708808,0.10864792129588199,0.108670021934492,0.10894474286527001,0.110288657937817,0.111101674094279,0.116272850302546,0.114904581185073,0.114915623477998,0.120063910455773,0.122236358684817,0.121434252386002,0.123337162719453,0.123895570047988,0.12523581104487699,0.12904460147255201,0.129584413684144,0.131222016024508,0.13121337963912699,0.132017546711126,0.13582851028131199,0.1360995290441,0.13692338282078501,0.13909235362049199,0.141806062822801,0.142610193776088,0.14317803009507399,0.14372054318369301,0.14533863941293099,0.149159264738602,0.14833368930330701,0.146718747441591,0.154040002356545,0.15185229397164199,0.15456673156797901,0.15403582462551299,0.155664022054567,0.155927684639659,0.15727603428741299,0.162733445269235,0.15891900828290401,0.16218726613133899,0.16082340349674101,0.16409830518160701,0.163255047630147,0.16734808830880599,0.16516238049923401,0.16761591257216901,0.169253960376649,0.17197136372056801,0.17141142335336401,0.17332564486348601,0.17686048689950201,0.17631598326852799,0.18092155649817401,0.18039306348963299,0.183912428657517,0.183371936210181,0.18473547778956101,0.18717774082432201,0.189623397011425,0.19233239071521099,0.19042899679233699,0.19314981736773901,0.19450955444277801,0.19857970990413601,0.20158584623252901,0.20374647561633799,0.20511179503899099,0.20809945664614801,0.208638211382114,0.21162618601810901,0.21543050575186501,0.21706683992411499,0.218711314852753,0.22142357529338799,0.222498688890751,0.22683696960473901,0.22820452638094699,0.23417727112053699,0.23336463621714801,0.23607981023389199,0.23825665290624001,0.24096706727936801,0.24694238246730299,0.244504447658207,0.248309572036604,0.248296964599488,0.25372738085721103,0.25699010250972099],[0.10241840461882901,0.100781510606541,0.104303793363724,0.103777981166259,0.104860523178742,0.107858510724368,0.109189190295528,0.109485243338987,0.110301965676629,0.112460361720073,0.112729163195257,0.11763154595472,0.117348024097763,0.117906234779976,0.122237835538822,0.123322578786148,0.12442264640037699,0.124424119241192,0.12579553871299901,0.128224213085633,0.13094751581919301,0.13148511475637201,0.133394456227171,0.13365903582623101,0.13473641888266999,0.138001690917573,0.13772923141949101,0.138824820313421,0.14099379111312699,0.14262202064770299,0.144512367689131,0.14697944427910101,0.146435725226817,0.149684264265045,0.15215135690777601,0.15050686993956799,0.15025062774321599,0.15485448332744201,0.15511242711645201,0.15673695849622901,0.157024222653078,0.15729225158914301,0.15918487210283799,0.15999416602535901,0.16517762460233301,0.16163640761363199,0.166532882957073,0.16326979610425199,0.16790045177285101,0.165971706527277,0.17006622405994201,0.16896452709047899,0.172234004948745,0.172786577098682,0.17550103074777901,0.17494403204901601,0.180117666336301,0.18120832101283901,0.181476081065158,0.18499690703428801,0.18500747777736001,0.18798704678641401,0.19016394564342501,0.19071226581831,0.19397049470435501,0.196958148285132,0.200210616236597,0.19667827241149999,0.20265921409214099,0.201301567889213,0.20726873429438,0.20892134997940401,0.21216672261053801,0.214347247384925,0.21679219123076199,0.21651717332390699,0.22004642498592999,0.22548339678989701,0.22765952712097901,0.22903447218040501,0.23310174215798601,0.23553628782793001,0.23716086736598899,0.24260155337751699,0.24178523035230401,0.24476959111049501,0.249117405157881,0.24803559153942401,0.255094556380346,0.25808074113349799,0.26216826768877799,0.26433705388173401,0.265961521050467,0.27328525411039001,0.27682264050901401],[0.10350536114056801,0.101870688429074,0.105661752595491,0.104321459427128,0.105404001439612,0.108399779724014,0.11000514812043,0.109485243338987,0.11166066132880299,0.11381684811102299,0.114087858847431,0.118717766056052,0.119521200720833,0.118993931735313,0.124411016175083,0.12577045226085501,0.12578134205255101,0.126054554023801,0.12823824119209101,0.13094161241636099,0.133665647557139,0.13447351278393699,0.136111847531519,0.13583073960848599,0.13691033593933899,0.14098861610432201,0.13990314847616001,0.14181321432779501,0.14398145272028501,0.145066199980801,0.14695654702222899,0.14969536274263401,0.14888137740072999,0.15239723303375699,0.153239794296711,0.15268078298304599,0.15269554349672201,0.156755920820078,0.15755734688314799,0.15863765956845699,0.15865465743568699,0.160008906473083,0.161629791869533,0.16352677873420099,0.16680732296453399,0.16625524042380599,0.17169667890850199,0.167890113794812,0.172518548162617,0.171676763451973,0.17522706630336099,0.17358632163504401,0.17712310003534801,0.17468727817091001,0.18093509298882801,0.179563597266407,0.18446402359563199,0.185011195998111,0.18718186638388101,0.19124543419347201,0.19288276163030599,0.19396531969555,0.19559210046845199,0.196960792977495,0.20130672684525699,0.204024118149605,0.206186667844939,0.20292533074305999,0.20944901025097201,0.211349295958011,0.21677888750514099,0.218430754730187,0.219502210304653,0.224398661568952,0.224124733243245,0.22276570048309199,0.23200074950340799,0.23553481097392401,0.23853133370511501,0.239628628204894,0.24179371222985999,0.244500749503408,0.251286167271695,0.25047831408446802,0.25237790149640599,0.25563917639383699,0.26243043335391297,0.26216531799395698,0.26813361611877001,0.275470572640509,0.27629134228050001,0.28171732396301602,0.28226072597327101,0.28686412204721501,0.28823421114645897],[0.10377636385059499,0.103230864948443,0.106477714433583,0.104865674108406,0.107578654916688,0.109214260694911,0.111364584206202,0.110300460730291,0.112475138286509,0.114905281486768,0.115717557209632,0.120346732011035,0.120878419519001,0.12062510695152,0.12468275931870799,0.12685740878259399,0.12686903499469801,0.128228467067279,0.13014115553873201,0.13202857295129,0.13556782548337301,0.13773217710112201,0.13882850241545899,0.13827712820280699,0.13908572583682199,0.143706751855458,0.14289007366290901,0.14507408389301299,0.146153892922948,0.147780645603518,0.15076237972870199,0.15458519826283401,0.15159876870507799,0.15457115009042499,0.15432675081844999,0.154311954186062,0.155681728249873,0.15920010015317501,0.16108996761837099,0.161624584755207,0.16245753242095901,0.16272556938340399,0.16624936110011501,0.166515172748576,0.170068192529751,0.170056642568262,0.172783635430241,0.170337250849111,0.17632364444868201,0.174936904623163,0.18038643169277299,0.17847762999606101,0.182284670672794,0.178761159879399,0.18691262145117599,0.18608386355602699,0.18935164584142,0.19071698533002401,0.19397313538352801,0.198036703193119,0.19831460257056099,0.20075733314198399,0.202382641074071,0.20347958642629899,0.20945227297463101,0.21189941002893001,0.214880110757629,0.212431785799021,0.21841640155531999,0.221128962985223,0.22574260473383101,0.226580735434768,0.230097847196337,0.23309285294161999,0.236351537324756,0.235533757511488,0.24205437696184801,0.247214446666147,0.25102767166902801,0.24940608597088201,0.25320899707486599,0.25917248539118498,0.263241968643179,0.26460138867618999,0.27085100742311802,0.27384500585765198,0.28063109583530399,0.281182674720658,0.28388933074113398,0.28769809708966698,0.28879284722683701,0.29557752456152703,0.296111337053721,0.29908503075227399,0.30371745021798002],[0.103233622010133,0.10295986223841599,0.10702045627404499,0.104866410528813,0.108667088292433,0.109485259391748,0.112723279858376,0.110843938991161,0.114378056646341,0.116263236705344,0.11735021329303399,0.120075729301008,0.122508117881203,0.12252507160334,0.126313198114507,0.129573327246126,0.12849726051608301,0.13121612466124699,0.13340129269673201,0.135014021284035,0.137196787425166,0.139091613186894,0.13991472251678999,0.140451049272665,0.14044368506858901,0.145063974666817,0.14560673256004,0.14670378225521399,0.14832854640002399,0.15022482493661601,0.15103559972633299,0.157301853146775,0.152955255096029,0.157830550828017,0.15649845058751499,0.15729814295240499,0.15758242932210201,0.16354792624013201,0.162448671296925,0.16434345692675001,0.167888636940806,0.16598570654140399,0.16869280401280501,0.17194627726842401,0.17196963002238699,0.17413274156435399,0.17767126168921901,0.175229299643725,0.18066926528760599,0.17819556492715699,0.18500600493654501,0.18309498996541901,0.18690423589018501,0.18446400352968101,0.192073467707785,0.19097369506303799,0.19478349079486501,0.196422782688317,0.19940791799222299,0.20428449393189599,0.20510292990176701,0.20890804625378301,0.20971665592736999,0.21108607281725,0.214071846218403,0.217873992809647,0.222216330858961,0.22112449630449399,0.22493887710616201,0.22819198315487499,0.23199261276021099,0.23799232613816501,0.24123474907447801,0.24205731461709801,0.24423124371333699,0.247216330858961,0.25319644582241302,0.25781599097385399,0.26325301093610398,0.260266081736164,0.270053893532953,0.27573828252795002,0.27764342620176602,0.278186151989467,0.28225889595852499,0.28633912252077098,0.29014198145328202,0.29422545267983202,0.29719644750795299,0.30155384706020999,0.30536232646564099,0.309429612495983,0.306437440063004,0.31240546328428598,0.31757467302933901],[0.10350536114056801,0.10404534591934,0.107291458984072,0.105408411935677,0.109481569263329,0.112746877416944,0.11408197952374,0.112744644076579,0.115193274037645,0.117892198647138,0.119796609913736,0.121707644950813,0.124410295807436,0.12415476996554101,0.12685667637537601,0.130387808217023,0.13067043713915399,0.13339003770472499,0.13638969072429699,0.13718720192029499,0.13855548709053001,0.14153726937399699,0.14072993990809499,0.14180900850443201,0.14343061025533799,0.14642341075258899,0.14778138603711599,0.148605956168257,0.150230724326258,0.153214699818186,0.154567476014767,0.15893081508856799,0.15485742900907301,0.15701606985712099,0.15840210536775401,0.15892858174820301,0.16084035721887799,0.16789501590668099,0.16516606661446301,0.16787386037436999,0.16951759888259901,0.17005884781629499,0.17195220876358699,0.17466441301956001,0.17577176858725099,0.17739287872235401,0.178758222224149,0.18147931569648601,0.184199664722035,0.18390061783866299,0.18935531189069699,0.18744282407875601,0.19152306468716901,0.190442276438817,0.19723282908400699,0.19423309178744,0.19885810892376199,0.201040138644486,0.207557146223636,0.211348974902792,0.21243547592744,0.21488262903450001,0.21786884187998401,0.21842229291858101,0.22276678605104999,0.22738339756043,0.231452515612113,0.23199408158783599,0.23607502651113499,0.23987530897551601,0.24123249968135299,0.24614230684274599,0.249113719042652,0.25455070288618997,0.25591308866678297,0.26297425474254699,0.26760086110220199,0.27221672415160397,0.27601593308759798,0.276834080107772,0.28363718400541399,0.286607875837673,0.28932306189398799,0.29122523179384102,0.295026952986921,0.29883176232988501,0.30263979225801002,0.30589845256200499,0.309150759985861,0.313505213856486,0.31650734903421801,0.323549733379694,0.32219023709606798,0.32924888690155701,0.33441808059384898],[0.10404810298103,0.10540404157151401,0.10810668038856699,0.106496108891014,0.110840268928693,0.113562094808248,0.11408197952374,0.114373610031563,0.116007018588134,0.120065379283398,0.121427048709535,0.123880085153476,0.124953774068306,0.12551272919730699,0.12875885430160999,0.13201898343323001,0.13284508660304001,0.13474726051608299,0.13584621246342801,0.13745894105073,0.140456924583166,0.14344092415423601,0.143175592082008,0.14316623532898001,0.145332784168382,0.14751037530070801,0.14941182483291501,0.15132261105219699,0.153492334325073,0.15538493477281601,0.15864209815685401,0.15974603247987201,0.15730160834217,0.15810450724605599,0.161659292830933,0.16273072432625801,0.164099761969661,0.17115662189230599,0.16788567921960501,0.17276739003618,0.171692252359676,0.17331898497429499,0.17656957274613599,0.17901225515927699,0.17848768705078399,0.181465283576838,0.18174662426490401,0.186098884927067,0.18827354643052399,0.18716149141707,0.193702409583627,0.193960152713125,0.19532667609284801,0.195878547941089,0.20321110600633299,0.20319901025097201,0.20592038464662599,0.20810536807536101,0.21570711087545699,0.21895546129374299,0.21759705459126599,0.22248986388542999,0.22764556322556501,0.227928007540945,0.231458764149305,0.236617377065549,0.24204665959703101,0.24258971446633101,0.246398167786026,0.24910781965300999,0.25128612713979298,0.25619150775235999,0.26161153787376001,0.26840057353304198,0.27004133425412002,0.27682853187227502,0.28226889883518003,0.28470715871268398,0.28905279159117903,0.291508033283152,0.29721826922984301,0.29937889861365102,0.30317513778887301,0.30290486748606399,0.30725153175444803,0.31132735986019999,0.31595060316643903,0.31839256119874298,0.32409420289855101,0.33007393660893097,0.33172181873607598,0.33685906743411698,0.34093733957271999,0.34555179800554098,0.35044774360787101],[0.104591581241899,0.107036697654916,0.10810594396815899,0.107581592571938,0.11111200805912801,0.114649051329987,0.115711677885941,0.11573230969692699,0.11736571424030801,0.121422602094757,0.122514741651681,0.124967782108813,0.12603999416963699,0.127142427559508,0.12930307299607799,0.13283420082453401,0.13393130670437101,0.13529000235654501,0.137746177115248,0.13936112299015399,0.14099966642362799,0.14479814696559501,0.145077029574644,0.144797414558377,0.147779176775893,0.15076977202510999,0.150771256905496,0.15295304583480601,0.15539450823811601,0.15701463313501801,0.16000079380902801,0.16056124987117701,0.158932779545187,0.16163712396808799,0.16356147477035701,0.16490389693613799,0.167359899127661,0.173602274066219,0.17141829995482699,0.177658698397196,0.174137168113181,0.17739286668278401,0.17874348578961399,0.18173038689722301,0.183920996818664,0.18554433226775099,0.187992954202436,0.19261843084904001,0.19289164282029,0.19341003462901599,0.19777850456652901,0.201568127997652,0.20048898315070099,0.20429510480687099,0.20701471741201199,0.20971706727936801,0.21352983678515799,0.216256077173969,0.222768646164722,0.22330328738069999,0.22466007074772801,0.23145875210973399,0.23606801145460801,0.23716640155532001,0.24177969616297301,0.24504056974138,0.24938287969836201,0.25209986767709203,0.25454739601743798,0.261333895340494,0.26649839159361899,0.27303344648924499,0.27682970171722798,0.27980995497521599,0.28444428070628303,0.28824083893012797,0.29340579670013001,0.29747524383341201,0.29720350871616802,0.30427241222270102,0.30781465056831597,0.30780429653751601,0.31594026117521001,0.31730633307103101,0.32137902085542602,0.32898821012999802,0.332249083708406,0.33550847641961801,0.33957965123129502,0.343115205608578,0.34529847339848802,0.35479017998034501,0.355874923227671,0.36213306398405798,0.36674620007069603],[0.105406062212796,0.10758017992897601,0.10810594396815899,0.108667812673269,0.112743183275335,0.11519327002445499,0.117344333969344,0.11736127163872,0.120081636717031,0.123052300456958,0.124414710316692,0.12605400221014401,0.12739869383500099,0.129316344616177,0.131205246909121,0.13419437333071299,0.13556247790738801,0.13773491811005101,0.13937587949063901,0.14071687296069699,0.14181488381493201,0.146428581748204,0.14616398609638301,0.146154637369736,0.14940813871768599,0.15294221222777299,0.15240095928088801,0.15512695887828401,0.15783795115080601,0.158374065207599,0.161629747724441,0.16328012605591,0.16083495345823001,0.164624045141648,0.16600639453705299,0.168707516368198,0.17089029856209001,0.17686314363143599,0.174135695272366,0.17901887892975599,0.17821326309608301,0.17983779046266901,0.18254415552685299,0.184447041781163,0.18826882290562,0.19016464995831101,0.193696534273126,0.195333612892165,0.199140910413073,0.199654879686162,0.20511326386661599,0.20591521766420101,0.20890995051254899,0.208913945643425,0.21353720500242601,0.21678154825026499,0.221135590768892,0.22467779700898499,0.22711499941086399,0.23335763520678701,0.23389479469963501,0.24178042857019,0.24557446651056999,0.246675798279722,0.24966013897196199,0.25862311576705399,0.25916180629197599,0.26459104267177003,0.27003284435018299,0.27302017085595598,0.27981437349766197,0.28417256966818,0.29014200553242397,0.29013459317006401,0.29639639194934803,0.29883645575586198,0.30481222042110201,0.31023963079934103,0.31133102590947698,0.31568105323127599,0.32139499535914701,0.32355708955738999,0.32952281522726401,0.33469251056833499,0.33686594202898601,0.343117188124553,0.34800555080393802,0.35126569197512802,0.35479409685401198,0.359414398491811,0.36241513707934098,0.37081839423269197,0.37163508847800197,0.37680036930981797,0.38223091198303299],[0.10622127960409999,0.108395401333471,0.10892190179306099,0.109211290934139,0.113830139797074,0.116280230559384,0.11815881494024,0.11926271314454701,0.12116785681836199,0.124954478383192,0.125229191287589,0.12768222773153001,0.12984214076088199,0.130675040268351,0.132291467010453,0.13555307299607799,0.13746391540002401,0.13882187463178999,0.14046210360516101,0.14261831045333301,0.144802541408899,0.147516278703541,0.147794420878991,0.14778728943994801,0.15239580433803401,0.15457191058997399,0.15593210316210501,0.15675665724048499,0.15892417125213801,0.16190446865521799,0.16244423270852801,0.165180827128139,0.16490956757393699,0.16706970534194099,0.17008027624554201,0.170881429411676,0.17496713397859001,0.179579798515376,0.17957048590744201,0.17983409632106001,0.17902847647419701,0.18255593022699601,0.18824995288514701,0.187981871777609,0.193429657122658,0.194780537086854,0.19749793641758201,0.20049592797639901,0.20457422820733401,0.20563019888728701,0.21109375807654501,0.21216743896499499,0.214071521149994,0.21706465474203401,0.22250239507193101,0.2227568634382,0.229831263008756,0.232556030556751,0.234991752091434,0.24123586072817199,0.244759953434151,0.25210432633143998,0.25426940634321599,0.25726773300341699,0.26460361800336402,0.26813178008423799,0.273559561682573,0.27600558306998801,0.28226036879934002,0.28225784851587399,0.29013754687807503,0.29666963602612001,0.30290786533916803,0.30046292550652098,0.30808267549119001,0.311332773653824,0.31459337232869999,0.32055835756497603,0.325454100501199,0.32952945505050402,0.33415937429869502,0.339582358128107,0.34582645472527501,0.34609967873609498,0.35370787675268101,0.35697665230265602,0.36457725134353602,0.36512000121037802,0.368649110404147,0.37299693649110399,0.37816865448005299,0.383318422325024,0.38223072135649699,0.39011045182421999,0.39119535760575003],[0.106764021444562,0.10948235785521,0.109737119184366,0.110025771905036,0.114917836752411,0.117911405775591,0.120603002299719,0.120348933245878,0.12415699527952501,0.12604217533852899,0.12685888964979,0.12985614077500801,0.13174284183311,0.13176199679009001,0.13392264222665901,0.13636903082098001,0.13882261105219801,0.139092877341817,0.141820062836927,0.14479222751000201,0.14697645846556801,0.14996045803663799,0.150239336632497,0.15077347820629,0.154028460421436,0.155930606242148,0.158105283798366,0.16056026864616499,0.16028360733791,0.16244868734968601,0.16461814976519601,0.168169961576111,0.16653852951573,0.17060084119677801,0.17306720143229201,0.17169517396216499,0.17686635819681301,0.18229645339931699,0.18147192340007701,0.18390871444995699,0.182832828313474,0.187175503470767,0.190966607769087,0.19368544783510899,0.19940644515140801,0.199943588591495,0.20157329899326701,0.20674299032114801,0.20946185446631099,0.213235216450613,0.218156766206627,0.21787471317729401,0.21923382820784701,0.22493994260816899,0.23092631614178899,0.229820607988689,0.23716896799047399,0.24179370420348001,0.24368445858371601,0.25101699658300902,0.25481136761817802,0.26052825944086999,0.26377807868678099,0.26868004006127,0.27682820479727099,0.28117381359662402,0.28306895840697499,0.28714617507654799,0.29367046659597001,0.29502370430942798,0.30344541611806403,0.30725862105499402,0.31214478651272698,0.31187082608149902,0.32058049833548902,0.32192839047955701,0.32953610290012397,0.337131526932199,0.34012214626055698,0.34555915819642702,0.350183166015406,0.357516440435107,0.35805622455436698,0.36049598536501898,0.36403175444797897,0.37245623335127997,0.37571488964208399,0.38033446689904599,0.38332155060681,0.38359108047602197,0.38956992325817102,0.390655398912714,0.39201557141889398,0.39554524647248601,0.39717288205490803],[0.108121244255921,0.110025836116079,0.11055233657567,0.11056925016590501,0.116275059563769,0.117640403065564,0.120058783605251,0.121435889767617,0.125243215380857,0.126041438918121,0.128216112461148,0.13148583913720899,0.13310080106487601,0.132848953311829,0.135279865038018,0.138543684298056,0.13990956757393699,0.14072331212442599,0.142634543807824,0.14587771119092499,0.148606897261367,0.14996045803663799,0.15241177683516,0.152403917002089,0.15538863694080601,0.157289305907512,0.158650975333649,0.16300518439966999,0.162998052960627,0.163533430597012,0.166789117127044,0.170614877329617,0.16898418168964299,0.17304723781747899,0.17714108314078,0.17658574590277401,0.18012428208039899,0.18419789089195199,0.185005280555708,0.185810896389381,0.186365445035507,0.18989290681468601,0.19477170004196201,0.19911802519577099,0.20239631200659799,0.204560208127256,0.206192872237039,0.20891543052381101,0.215710389651877,0.219212752939341,0.22195743594386499,0.22439571990051099,0.22629683633792899,0.232005172039044,0.23880602654356101,0.240144485683987,0.24776239160774499,0.25021247033048499,0.25319164604689498,0.26324304819135103,0.26622738888359099,0.27302606623240699,0.27627810276592302,0.281174148698009,0.28715062169132499,0.29149769931830299,0.29665223282667602,0.29937372360484599,0.30290665134912198,0.30806205170658502,0.31458452725742803,0.31975423865126001,0.324912859593884,0.32573029828598299,0.33416452924154899,0.33496745021798002,0.34692005508344398,0.34772790024429101,0.35615109292012198,0.35696337666936701,0.360237529894254,0.36648163050461202,0.36946191586812199,0.37625762345616598,0.37680054789678302,0.38332064763900803,0.38739747502912802,0.391198148779557,0.39228525980911999,0.392555526098739,0.39554449399931702,0.39473149389561701,0.398534372894078,0.39853363647367102,0.39961853422882099],[0.109479939908095,0.110569314376949,0.111910295807436,0.111927949831269,0.116818537824639,0.118725886746487,0.122230487387506,0.123882286388318,0.12687291775624801,0.126856656309425,0.130117549953784,0.132843798368975,0.13364427932574599,0.13475039080446499,0.137725525238311,0.14044586222429001,0.14154000235654501,0.142625486037469,0.14562441467620499,0.146964667712665,0.15132355214530699,0.15213437910649699,0.15349873335689901,0.155121312319627,0.157563294431073,0.15891826784930599,0.16028215456304501,0.16409287734181699,0.16516975674288201,0.16570808407408799,0.168148557226006,0.17115613830288301,0.173329798515376,0.175764633135018,0.180399747457965,0.17903066165627901,0.18392863793286701,0.18745802403676201,0.18907842584378901,0.189340563416593,0.19043858631039801,0.19587043929022399,0.19830578960480999,0.204005647441559,0.20728540709320101,0.209455951063479,0.21379789782674499,0.21625387995231701,0.22086901862088201,0.22355690093744901,0.22739370343294699,0.23010151324561501,0.23363158359844499,0.238794235790658,0.246682046816914,0.24938287969836201,0.25428266592374499,0.25917987367440298,0.26460174384352497,0.27492562153882399,0.27546654339751597,0.28199051586831497,0.28361434293320498,0.292312507364204,0.29828972881749899,0.30127811079230299,0.30697611052197499,0.31105704942548801,0.31350005891363297,0.31893016414911202,0.32653665856644398,0.33334270413252598,0.33632001572207398,0.34230273123279698,0.34665939035145599,0.34963620831860498,0.354798300670781,0.35940533470167102,0.365934474154894,0.367560478375486,0.37219187046449298,0.375988830007362,0.38223663679890002,0.38549529308970398,0.38983960763520698,0.39174309988124201,0.39282857553578499,0.395816985602921,0.39581713208436398,0.39798957228702703,0.40043434155908902,0.400981501921997,0.40125029537080098,0.40179156035725699,0.401247496170614],[0.110295157299399,0.110298311666922,0.1132689954728,0.11301416993260099,0.117361279665101,0.119268628586949,0.123316707488838,0.12442650508278599,0.12823309026242699,0.12767039684672399,0.13066029179424599,0.13420323044155699,0.13500297899111,0.13692430786113299,0.137725525238311,0.14153134590521399,0.143713178979616,0.14452692353010499,0.146980897053965,0.14913858878252301,0.152138033116204,0.15512204071365401,0.154856692588665,0.156478535130986,0.15783429714110001,0.16054722979109901,0.16245533118611599,0.16490809473312101,0.16870236945172401,0.16788200113075599,0.17059273655910401,0.173872797200013,0.17740514905149099,0.180110994407539,0.18202870939975799,0.181747316540219,0.18637355769956199,0.19099063273241401,0.19152260517688699,0.193415177532299,0.19451245999250699,0.19994801915351201,0.20237967131329901,0.209166485671787,0.212717980440674,0.21597548093269101,0.21895799562337501,0.22087123992167501,0.22657628881999001,0.229533692979389,0.23472624143224,0.237704317534527,0.24232355367031899,0.24531082800461901,0.25564576805879402,0.25916401555319901,0.26433038997935299,0.27059220481139801,0.27737053729232902,0.28552123836455801,0.28579633653521702,0.29203898035752102,0.29502223949499301,0.30263564863909498,0.30589549484080297,0.31213737013717702,0.32001664310121403,0.32572656401247102,0.326812330623306,0.33278593017241598,0.339037431105563,0.34475280995553698,0.35072000043984602,0.35588454083806298,0.36159401628520499,0.36539192294096901,0.36729021609905699,0.37353949974460099,0.37570824981884499,0.38114893583037202,0.38387372745750897,0.38766921415956301,0.38848074944882799,0.39337426305787798,0.39527291740308701,0.39690541095228499,0.397446671925551,0.40016334286225202,0.40097796630140198,0.40233592553316799,0.402065512762105,0.40315173687662698,0.403424948847877,0.40260604132815397,0.402335189112761],[0.110566896429834,0.110842530361389,0.114354479153724,0.11410039003393201,0.11844823618684,0.119268628586949,0.123588446619272,0.12551346561771501,0.12877582808969901,0.13011605704701801,0.13120451048871401,0.13501697499204601,0.13608919507925099,0.13773952525243799,0.139353754772887,0.14180308503564801,0.144527659950513,0.145884882761871,0.148339596719329,0.15185451125924601,0.15404021104243701,0.156208997235393,0.15703134205255101,0.157564755232317,0.15946325908289299,0.16136023792118001,0.16462777138877899,0.16653779309532199,0.17250820215819701,0.170870399158322,0.17412166315271699,0.17577349827224101,0.180937021326735,0.183913128959213,0.18447362515326399,0.186093681825931,0.18936047887312199,0.193978290326382,0.19505374504491399,0.19776153077844,0.20021825735080001,0.20592629607583801,0.20754273084432101,0.214328068348803,0.21842229291858101,0.22140880274014199,0.22466453341526699,0.22874874507541401,0.23472627353776199,0.23524095916530799,0.24341970441088101,0.24694346402207101,0.25128873571344401,0.25917102458994001,0.264607280039451,0.26921615411806299,0.27492749971185299,0.28145515027149998,0.28850963237893201,0.29367267585719298,0.29938332917566801,0.301817194609868,0.30670188722678599,0.31377253446447501,0.31839109237111801,0.32654032461572102,0.33033978437610501,0.33985924417501501,0.34311299634735498,0.34827729796075402,0.35071706679778603,0.35643096879375402,0.36077141462387302,0.36484899850035102,0.373006331369439,0.37517084953458202,0.37869738025362698,0.38250543426089401,0.38467492476873599,0.38875617068130203,0.39147579131282401,0.39418579834714401,0.39554450202569802,0.39799310389443199,0.399892482620478,0.40097707537317101,0.402061822633687,0.40070755754352899,0.40288014021444601,0.40396636031577698,0.40315246928384502,0.402879261325785,0.40260899102297498,0.40124734567598003,0.40016274891009801],[0.110837899139861,0.110841053507384,0.114354479153724,0.11545908568610599,0.119534456288171,0.119812106847819,0.126305105516403,0.127959125818009,0.12986278461143799,0.132017494539653,0.13283420885091399,0.13556045325291499,0.13636093420968601,0.138283003513307,0.14071171400465299,0.14316104426741499,0.14561461647225199,0.146427624602333,0.149699032805101,0.15321321493779999,0.15594313341545901,0.158381441451247,0.15920451867562199,0.159467669578959,0.16082269516866499,0.16380515367468601,0.166529949315013,0.16816822787793101,0.17604229172104499,0.174130532303131,0.177926759438783,0.179301688445447,0.18392541534111001,0.18717401056400099,0.18664532893551899,0.18962556212755599,0.194524266798171,0.19805216802167999,0.19940157514506099,0.20156294094927699,0.20483857102816999,0.211087866713284,0.214063741580728,0.21786215791165101,0.22358459997643501,0.22629716541952799,0.22928410665903801,0.23472627755095199,0.242058815550245,0.24339388153833,0.253739179636495,0.25591235625956499,0.25916401555319901,0.268406484962255,0.27710287355657598,0.27926902910333501,0.28688035540170598,0.29096161335384202,0.29991899375515502,0.30181822198656799,0.30888905985079301,0.312952619634003,0.32055913411728598,0.32328266760928498,0.33088152693219902,0.34012065335379099,0.345554966419229,0.350182409529047,0.35262460233297999,0.35832797572437303,0.362124955333193,0.36512516417961199,0.37082503806912198,0.37245548489130198,0.38115779294121599,0.38304907505596802,0.38548791283286599,0.39092492878192597,0.39174089463320899,0.39445754550395901,0.39690541897866599,0.39853141918606699,0.399348117444567,0.40152129406763798,0.40288014021444601,0.40260677774856202,0.40287999774619199,0.40125103179120902,0.40206492282314099,0.401793183692707,0.40043507396630701,0.39989086329821999,0.40016333483587202,0.39907637831413301,0.39907652880876598],[0.110837899139861,0.112743967854025,0.11571243838549,0.115730088396133,0.12034967367947599,0.12062732423912299,0.12684932421087,0.129317821470182,0.130677265582335,0.132561713234121,0.13392190580625199,0.13664815020825199,0.13717689203458799,0.13936996003504601,0.14098271671467999,0.14397700209231701,0.146158094733121,0.14860153764581099,0.15187147300776399,0.15375595677826201,0.15567139428502499,0.15946692513217001,0.160018999646518,0.16109736794116,0.16326761092216999,0.16461963464558199,0.168161860951627,0.171971102863202,0.17848720747455099,0.17820441401161999,0.18010141692904899,0.183919048414806,0.18664207022505,0.18907471163622899,0.18936198381945901,0.19342844112601801,0.19724092168211099,0.20239704842700601,0.20320224086910901,0.205637563091364,0.21000162253281099,0.21598066396787699,0.220852805332342,0.224382432227651,0.231191086367385,0.23254274689708199,0.236619606392723,0.243689258359234,0.24993261853599499,0.25072421027640002,0.26243189014196699,0.26405789837575,0.26840167314716601,0.27845863556668998,0.288236817713515,0.291495817132084,0.29801725727984701,0.30427907211189198,0.30915886061034498,0.31186815129020901,0.32056501344097699,0.32300550665884598,0.33251493950195998,0.33741015671026298,0.34446259209067598,0.34908806472408999,0.35696653705667503,0.35942156404297099,0.36213178979615901,0.36647426630053498,0.37027419159098601,0.37734827411951399,0.38087278219068199,0.385223561985649,0.38712721275269901,0.39065556144691899,0.39309810139180601,0.39635825058937701,0.39717346798068098,0.39907490146012697,0.400706821123122,0.40097707938636101,0.40206847048330702,0.40315173286343697,0.40260619182278801,0.40314804674820798,0.40152129808082798,0.40097855222717599,0.39962000706963602,0.39989174620007101,0.39989159570543698,0.39907637831413301,0.399348117444567,0.39907637831413301,0.39907652880876598],[0.11138137740073099,0.113830187955357,0.11625518022595201,0.11708657478708399,0.12116489107078,0.121444022497623,0.128208019863044,0.12959030103421501,0.13040552645190001,0.13283345236455599,0.135553081022458,0.13800537301961099,0.13799284985949001,0.14181340294773601,0.14261315149728901,0.14506469904765401,0.14724505125486001,0.149960233297985,0.15241347842781799,0.15484217687959401,0.156759087227171,0.162185060883306,0.160834217037823,0.16327054456423001,0.16462778342834999,0.16924068875654999,0.171150254966002,0.17550297513844701,0.18039086626798001,0.181463810736022,0.18444925104238599,0.186636443732344,0.18854277129727801,0.19233558521463701,0.19397860335521999,0.19614288674873501,0.20023079656368201,0.206201396253093,0.20836530040013099,0.21107161730603199,0.21706611554327801,0.22059802393723499,0.22547090172210801,0.22927153534063499,0.238253358077059,0.240146291619592,0.24449415382526099,0.25265370799514197,0.25699489827204902,0.261593799572932,0.27221304606275498,0.27465205440023799,0.27790959703075302,0.288513015498299,0.29720497754379299,0.302905178508307,0.30997158179732498,0.311344313582337,0.32029648285613299,0.32491089313067001,0.32735702688741097,0.33686054027493201,0.34582722726439402,0.350180422999882,0.35532700236521397,0.359142436629318,0.36457449628844102,0.36675854063066199,0.371641186520561,0.37543798352922497,0.38087352663746998,0.38440908101475302,0.38793875205515499,0.39255536356453502,0.39473370315683998,0.39554686579474502,0.39717419236151802,0.39934738102416001,0.40043434155908902,0.40206403992128997,0.40288073416659997,0.40314878316861602,0.40396769871471999,0.40315246928384502,0.40152070814186402,0.401519821226823,0.40016259841546398,0.39989159570543698,0.39934826793920097,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.111653116531165,0.113830187955357,0.116797922066414,0.11925975542334501,0.12116489107078,0.12225850346852001,0.129023237254349,0.13094900069957899,0.13176127642244301,0.134462414306349,0.135824079719295,0.138276375729638,0.138808067250795,0.142899623049068,0.14369863517821299,0.14560891372893101,0.14996244255920799,0.151589931660186,0.153227959398715,0.155384918720056,0.159205483847872,0.16327202141823599,0.163279132791328,0.164901727806817,0.16625822222414899,0.17195734364049001,0.173868386703948,0.17876237186284899,0.182835053627458,0.18336377538784299,0.18635142896862,0.18935162577546899,0.192073907152115,0.195323250834984,0.19805544278491,0.19858853892264799,0.203489460880866,0.209734004948745,0.214340615588066,0.21568971369579801,0.22222769822029501,0.226577041293159,0.23198969918410201,0.23606355681345001,0.24531415694591699,0.24911222613588499,0.25509053516373298,0.259985517600408,0.26433261930652702,0.27165334648696599,0.28172173044589099,0.28416294804459702,0.29013270295746402,0.29856811981074399,0.30806866343749301,0.31105440673972001,0.31948248346806402,0.32438192054589698,0.32898845292800799,0.336320990927301,0.33985188799731902,0.35071705074502502,0.35397941723019899,0.358055702839637,0.36511332928161599,0.365926337411698,0.37082302344762602,0.37626868581504203,0.38060342288205501,0.38386191262546399,0.38848148988242598,0.39038513663628499,0.39337281028301302,0.39744668396512201,0.399351807572986,0.40070696359137498,0.40152129005444798,0.40260752219535001,0.40315247329703502,0.40396621383433401,0.40342420841427901,0.40287778045858902,0.401248822529986,0.400706813096741,0.40070549075056,0.39961985657500199,0.39907637831413301,0.399348117444567,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.1119248556616,0.114645405346661,0.11869935955905001,0.11925901900293701,0.12279458943298099,0.12334546400344901,0.12983698180483799,0.13149321939404701,0.13230549110372,0.13554937082808799,0.135824079719295,0.13881911757009999,0.139623284642099,0.14425831870124201,0.144787804974365,0.147510351221567,0.15077765995051301,0.15322036644279499,0.15512866047094301,0.15701535751585399,0.16083444578966599,0.16435898195316501,0.164908831153529,0.16734369787869199,0.168975625568067,0.17386025798713101,0.17685751713872999,0.18039280664545801,0.18690893132275699,0.18662317211224499,0.18825507973566899,0.19288275761711501,0.19370360551431601,0.19994135123794099,0.20212785165258401,0.20293563260238701,0.20756554783738801,0.21543979026746801,0.21760001632565801,0.22139256537246099,0.22548709895788699,0.232284311492268,0.23660484989223801,0.244215738752874,0.25509308353953097,0.25780494065454801,0.26351225098555903,0.26813402546417398,0.27302756315236398,0.282252661467499,0.29231588647037998,0.296383852736466,0.301539855072464,0.30780359222262998,0.31893383019839,0.32137902085542602,0.33116432842151,0.334703613059114,0.33930864852126802,0.34963399905738202,0.350717787165432,0.35913728569965497,0.36266770118684499,0.36756215388240798,0.37353578955023098,0.37326036029137699,0.37870198538941902,0.383600495420308,0.387123689171674,0.39228436486769802,0.39282932399576298,0.39663293540144301,0.39853585376127398,0.40125029938399098,0.401252504632024,0.40315187934488,0.40287704002499097,0.40342273958665398,0.40179377764486102,0.402608254602567,0.401794510052078,0.401248818516796,0.399619120154595,0.39961985657500199,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.112740073052904,0.11546062273796499,0.11897036226907701,0.11953149455378,0.123338067693851,0.124433160958786,0.13065072635532701,0.13258091634938399,0.133935929899519,0.136907330059854,0.13718204296425199,0.14017486754064301,0.14043776561299601,0.14452932141126901,0.146145027785724,0.14914079001736599,0.152135619182279,0.154307322964534,0.156759095253552,0.15918706129811,0.16192066990418799,0.165987943894958,0.16653926593613799,0.17006035677582201,0.17332272727418699,0.17657691287107199,0.17848795593452901,0.184197154471545,0.18799662827809399,0.18825360689485299,0.19097173863279901,0.19586968681705499,0.19832169789089199,0.20374423023640301,0.20538798479739301,0.20673703876003299,0.213267651054072,0.21951440438317399,0.222221066423435,0.22682292945871099,0.23146537588021299,0.23853431550545801,0.24665699648348199,0.253175777892716,0.26270104277129702,0.26813327299100498,0.27274918018549899,0.277644911082152,0.28199570291669002,0.29339398588127602,0.30263903175846102,0.30752075060141698,0.31077824908683899,0.32166083911312998,0.32762581230983501,0.33034567573936602,0.34094327106788402,0.34556581808561798,0.35397887946270801,0.35724122186874002,0.36076477078744301,0.36538360761080702,0.37054888844262202,0.37571359137504401,0.37869958148846999,0.384398735010333,0.38685047720042398,0.39093082817156799,0.395002651113468,0.39554376961847998,0.39717640964912199,0.40016407125628001,0.40179377764486102,0.40233799232613798,0.403695955571095,0.40315114292447302,0.403421262732649,0.40179377764486102,0.40070608068952401,0.40124955493720299,0.39934885787816499,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.11355529044420901,0.11546062273796499,0.11978632009397901,0.120347452378682,0.123609806824285,0.12579038377014501,0.130922465485761,0.13339466089987301,0.13447867173998099,0.13799281374077799,0.13826973991958799,0.14153356319281701,0.141526203001931,0.14588728064303499,0.14750299103067999,0.15104370436400699,0.15322257570401801,0.154850064804996,0.15893227187662201,0.161359505513963,0.165182287929383,0.16843581335647501,0.16844070342877299,0.17114657687715401,0.17522416878001301,0.179023305478583,0.18120461483166,0.18772902674678901,0.18962559021988801,0.18988477809786999,0.19341665839949501,0.19940156711868001,0.20103835277483201,0.20727315683001599,0.21027930118478999,0.21080944762770701,0.21978719296285501,0.226577412513256,0.22684063565401699,0.233071464644276,0.239618302266425,0.24776461290853899,0.25508166200012899,0.26051053317961198,0.27139522210439498,0.27411006503294499,0.28117310125535799,0.29041223570333102,0.29557826098193501,0.30589547878804202,0.31323540507055297,0.31757585491386298,0.32110139036173002,0.33116729015590102,0.33577727388161199,0.34338473547779003,0.35072516742227,0.353986061066629,0.36185636856368603,0.36403396370920199,0.368095847985491,0.37054739553585597,0.37870181884202397,0.38304686579474501,0.38576407449893702,0.390919009326333,0.39282800164958198,0.39554745172051903,0.39962000706963602,0.40125029938399098,0.40016554409709498,0.40288221102060601,0.40206478035488802,0.40287852089218701,0.40260751416896901,0.40125044185224501,0.40124955895039299,0.40098077352797001,0.39934885787816499,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.113827029574644,0.11546062273796499,0.120600801064876,0.121705411610448,0.124969242910057,0.12714907942231901,0.13119420461619599,0.13448014458079699,0.13529388913128601,0.138264552871213,0.140171913832632,0.14316325754182799,0.14397259560944101,0.14643149933750299,0.14831821243517501,0.15294514988302399,0.15458127135619201,0.15566528219629999,0.16110471609247501,0.162989203876164,0.16599603247987199,0.16925546532298599,0.17197331212442599,0.175489984441664,0.17848430192482201,0.18119500926083801,0.18500970710453499,0.189359461529398,0.19098428988525201,0.19232896144415801,0.19586009729899501,0.20238701545142401,0.20511443973135399,0.20971881301712,0.214624922023714,0.21733414447934299,0.22331908129086001,0.23228172499116301,0.23254273887070101,0.24068017634920999,0.24559362146755001,0.256461766015599,0.26241567284023698,0.270022167257569,0.28036261340874302,0.28226300345872801,0.29122673272698801,0.29937890664003203,0.30806500541459603,0.312955549262873,0.32464477848634699,0.32790419126351,0.33006730882526197,0.34067817978706999,0.34338229545812798,0.35343171909979998,0.35887440769325402,0.36213751862521598,0.36729041475197399,0.37028101802757202,0.37516108142955601,0.37896985179127901,0.38413808231791602,0.38848017556262499,0.39065096433750701,0.39553932701689298,0.39907652880876598,0.40016333884906202,0.401793183692706,0.40369300587627399,0.40315246527065401,0.403153954164231,0.40260530892093699,0.40206329948769198,0.40043360112549098,0.39962000706963602,0.40016259841546398,0.39934885787816499,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.114370507835513,0.117091057520574,0.121145019759343,0.122793108565785,0.12605546301138901,0.12796356039321599,0.132281161137935,0.13475188371123101,0.13583736739215499,0.13880729471167499,0.14126034720837699,0.14424947764315901,0.146147253099708,0.14914594496021999,0.150763868622278,0.153216885000268,0.155124013196654,0.15702324142806601,0.16191772422255701,0.165161648092017,0.16708225659439399,0.17034021258350199,0.17306100506657199,0.17793195050034799,0.18174443908282201,0.183369666751104,0.18827131710334999,0.19017394250029501,0.19261620553505601,0.19613332130981501,0.199666670439065,0.20619063087029399,0.208375309296571,0.21406886040487,0.22086903467364299,0.22141097588265299,0.230383582327708,0.236899080947331,0.23987601730359201,0.250461332269998,0.25428413073818001,0.26515670584824602,0.26784824618771003,0.27844535592021002,0.28932853187227497,0.29258542436597201,0.30371938056248199,0.30997527995212398,0.31512654873024298,0.32219248648919402,0.33279327832373201,0.33795781872195002,0.34120787675268099,0.35018832497145003,0.352344539846003,0.36130994462118499,0.36538952305320899,0.36974474946295499,0.37598459408507101,0.37815924354895702,0.38331474022298601,0.38657709466858903,0.39119962162037197,0.39472722988099501,0.39499732561002898,0.39907268417252401,0.40260840108401102,0.40342273557346398,0.40288014021444601,0.40314952360221401,0.40260898700978498,0.402882955467394,0.40070534025592602,0.40097707938636101,0.399890122864622,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.11491398609638299,0.117362060230601,0.121416758889778,0.12333658682665501,0.12659894127225799,0.12850630223367701,0.13309711896283699,0.13502288642125801,0.136653325217057,0.13962103926216399,0.14234730774330601,0.144792219483621,0.147505948751882,0.149961902785122,0.152936312838131,0.154843637680838,0.15621096971839299,0.15811093437021301,0.16463364669927899,0.165703649498881,0.168169213116133,0.17197065539249201,0.17550592082007799,0.18282474374175101,0.18473357754398501,0.18418562457600601,0.19017423546318199,0.192618121833392,0.19533064714458301,0.19803697609005499,0.202113067059767,0.20863407378298399,0.21408035819488599,0.220046388867218,0.224128431398045,0.226031293573213,0.23446042175739801,0.24287586897608099,0.246123816068749,0.25834325594618301,0.26216532200714698,0.27412263233815798,0.28061484241486201,0.28659237890359002,0.29802050194414997,0.30481148400069502,0.31241284354112397,0.31894195088882499,0.32219325902831403,0.332248331235238,0.34093956889989502,0.34746353735750402,0.35370566749145699,0.35833830166284097,0.36103799078507298,0.37000486037469099,0.37326702018056801,0.37490263398517198,0.38359181689643002,0.385222988099446,0.391465453334785,0.39173941777920301,0.39880463517050802,0.39934605867797801,0.40043211223191499,0.40179229276447498,0.40369535760574998,0.40369521513749701,0.40342435489572298,0.40206330350088199,0.400977815806768,0.40098151797475801,0.399619120154595,0.399619120154595,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.11491324967597499,0.118178018055503,0.121416758889778,0.12360832595709,0.12768516137359001,0.12932077919138399,0.134182598630571,0.135567101102536,0.13746780618795401,0.14016378110262601,0.14316252513461,0.14642339871301799,0.148592168853213,0.15186556157855099,0.15429649337069101,0.15647259962263199,0.156754447979262,0.16001310828325699,0.165448127670176,0.16869204351325601,0.170072131475965,0.175231524957709,0.17795083657358299,0.18472692166798499,0.18527705981804499,0.186902279459947,0.19207493653541,0.19479129845646301,0.20049148537481101,0.200483364684375,0.20455871923367999,0.212164477230603,0.21869845057146201,0.22439496341415299,0.22901826691824601,0.23364148213215299,0.24152342988747999,0.24993814068575501,0.25508901015144497,0.26459104668495997,0.27112977164305502,0.28145517435064199,0.29257063977315501,0.29881844255150303,0.31105808884175801,0.31377888333142301,0.320294030796901,0.32790639651154202,0.33088892324179697,0.33876492344919901,0.34881559719962801,0.35615919755779701,0.35940997996936502,0.36295271595056899,0.36837570379317103,0.37353746907034302,0.37788584897755201,0.38196638254885101,0.38820990927300603,0.39011134676564202,0.39608649941937202,0.39608872473335599,0.40016185798186599,0.40315040650406497,0.40260825058937699,0.40342346798068102,0.40315187934488,0.40260751818216001,0.401522180982679,0.40043360112549098,0.39989085928502899,0.399893817006231,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.115727730646872,0.11872149631637301,0.12277545855514201,0.124422806927986,0.128227903214052,0.130407735713123,0.13363912438289199,0.13638305892743799,0.13936998010099799,0.14152174033439199,0.14452122479997501,0.147237879683915,0.15076461306906599,0.15240829940582301,0.15483997163156099,0.15864652069249,0.15892836102274099,0.16273049958760499,0.16734956516281199,0.16977826762777801,0.17360327535718201,0.17794965669565499,0.181753711558855,0.18744431297233199,0.18744950002070801,0.19043268692075599,0.19533434128619201,0.19777969247083799,0.20375088209921299,0.203743501842375,0.20890508451939199,0.217323838606826,0.22168684458583701,0.23308990324679901,0.23200667297219099,0.23798857581189201,0.24749801266819699,0.25618740426534697,0.26188322884591198,0.27410342119651498,0.27954558807523899,0.29069137114336402,0.301538059169835,0.309413334996399,0.31839430894308901,0.32138464132834699,0.330341758865699,0.335783161231683,0.34066860230857998,0.34691784582222102,0.35588083465688303,0.36457943652561797,0.36674546365028898,0.37055921438109102,0.37217710593762698,0.38087221633085899,0.38359017349503,0.38875765957487801,0.39282947449039701,0.39418743372216303,0.39989085125864898,0.40097782383314901,0.40206403590810003,0.40315040650406497,0.40369669199150199,0.40342273156027397,0.40179392011311399,0.40124955493720299,0.39989174620007101,0.39907637831413301,0.39907637831413301,0.39934885787816499,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.116814687168611,0.11926571501084,0.12277545855514201,0.12605177288297001,0.12985760157625301,0.13122221668402001,0.13445434578738599,0.13665479805787301,0.14018520150549199,0.14260869685613101,0.14560670848089799,0.14886831446652299,0.151581311327567,0.152680042549448,0.15538271347202301,0.15973274079382199,0.16137401319665401,0.16408772239896299,0.16816478255411599,0.171949234989625,0.17523150087856801,0.18093731428962201,0.18338193708024,0.18934575046496799,0.18935167794694199,0.19233560528058799,0.198320526039344,0.19940939083303899,0.20483931948814799,0.20646016475269599,0.214064453921994,0.222481731155423,0.226848415223283,0.23798048722697901,0.23879500832977801,0.24505158394197399,0.25510746079353902,0.26515258630847199,0.27193538346353702,0.28089470624892199,0.28742013550777601,0.29992904077690202,0.31131774224980702,0.31729158058373602,0.32572831978319799,0.32926508815052602,0.33876569598831902,0.34393240952904702,0.35044606408775802,0.35479238924156797,0.36185910355282902,0.36892211168291,0.370278808766349,0.37734754572548701,0.378424900689595,0.38739321904088603,0.39119742038553001,0.39445976680475198,0.39690408860610299,0.39934826793920097,0.40124733764960002,0.40342273958665398,0.403150256009431,0.40369388476493501,0.40315394613784999,0.40206477232850801,0.40043522446093999,0.399619120154595,0.39934826793920097,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.117086426299046,0.11953745414127501,0.12277545855514201,0.12659525114383999,0.13094308525717599,0.131493955814454,0.13472534849741299,0.13719901675234,0.14072794334595401,0.14342539511463101,0.146150186741768,0.149955275001453,0.15130957219713201,0.15403873820162201,0.155654452602458,0.160276219054691,0.163004447979262,0.166260899022034,0.16870826081498599,0.17466663030716301,0.178221375760138,0.18365470960715999,0.18555511370331099,0.19124719197079401,0.19125533272718101,0.193150086251484,0.20049518352960999,0.20266805113703301,0.20782846196250099,0.21243474352022301,0.21759558977683099,0.22655192273549299,0.230108548368092,0.243413064587642,0.24639782064507099,0.25211387973078903,0.26488270929830499,0.27085689878637897,0.28144185055907001,0.28985915989802002,0.29856369727510801,0.30998488150975601,0.31974019047885099,0.325985783995975,0.331160156710263,0.33605268307451502,0.346103400970036,0.35153669468515603,0.35560763472520401,0.36239372470285502,0.36783515917436199,0.37218521057530202,0.37815703428773401,0.38278012709934001,0.38766257834951301,0.39065408860610301,0.39663220700741503,0.39853217165923599,0.399349740780017,0.40179244727229901,0.40314951958902401,0.40315099644302899,0.40233577503853502,0.40287866737362998,0.400435814399904,0.40043433754589902,0.39934826793920097,0.39907637831413301,0.39934826793920097,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.11708716271945301,0.11980845685130199,0.12304646126516899,0.12686699027427401,0.13230030806853499,0.13258091233619401,0.13581230501915301,0.13774102217239401,0.14099894605598101,0.14505582989724,0.146421189451795,0.151585717810442,0.152669008282904,0.15539743786698601,0.15646893357335401,0.16163565514046299,0.16354792624013201,0.168705814775539,0.17034090887200801,0.17684054335064101,0.18121051020811099,0.18555467425898001,0.18881303758689799,0.19260515120256,0.194245207608752,0.19505152775730999,0.20429732209447399,0.20565497231059299,0.21081611955646901,0.217052839909989,0.220583983791206,0.23171350139931901,0.234999852715919,0.248031897397815,0.25264413452984202,0.25917541903324498,0.27140002589310303,0.28199157535053598,0.289859876252477,0.29909830638556401,0.309157853299597,0.316776158535783,0.32435608162058399,0.33522788419152999,0.33713620831860502,0.345288879867237,0.352623675286036,0.35805401529314401,0.36294386285291602,0.36755530737987202,0.37109161021713299,0.37843374977405703,0.38358813479439102,0.38875692716766103,0.39364454540025801,0.39527291740308701,0.399619128180975,0.40125178025118702,0.40179465653352198,0.402335189112761,0.40396473698032798,0.40233651547213301,0.402336511458942,0.40124896901142898,0.39934885787816499,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.11844512195121901,0.120623674242607,0.12386315551047899,0.127680734824763,0.13257204719897001,0.13339612972749801,0.13689926154089199,0.13801350173642701,0.14181416746047601,0.14587031086813701,0.14723714727669701,0.15267415118618699,0.15402696751466999,0.157841621213274,0.158099372369153,0.163536356212691,0.165993578414045,0.17006451042771301,0.172784351784698,0.17819850258240799,0.18501708334818201,0.18745684817202399,0.190442735949099,0.193691371303892,0.19587564640455099,0.19912835916061999,0.20728498370163201,0.208914369034995,0.21326029888956599,0.22221589141463,0.22710277724000999,0.23633086136867801,0.23798824673029301,0.255365175830706,0.25997963426352699,0.26814282639035403,0.279281221175261,0.28932632261105201,0.30154172923230199,0.30942294056722103,0.31703755968817798,0.32356891241581498,0.32924666158757299,0.34093515840382899,0.34610139036172999,0.353980861978683,0.36186281374719897,0.365117035462796,0.36810691034436699,0.37380680028663799,0.37897132061890498,0.38522502278689402,0.389838134794391,0.39500324506562201,0.39744815680593698,0.39744462118534202,0.40288073817978998,0.40260899904935599,0.40315261576528799,0.40287866737362998,0.40369299784989299,0.40043433754589902,0.40043507396630701,0.39934753151879399,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.11953207847295901,0.12198236989478101,0.12495233333301201,0.12904017091053499,0.133116265893437,0.13339539330708999,0.137442739801761,0.13882798270732399,0.14371634538670899,0.14722753367949501,0.148867582059306,0.153761107707926,0.155384922733246,0.15892858174820301,0.16027255300541399,0.16489579229846299,0.167624749617061,0.171965947920349,0.17604227566828401,0.17982746452420101,0.18746052626087201,0.190172030215149,0.19261664899257699,0.19477832782563101,0.19886330399851801,0.20293418785390299,0.20864147009258299,0.21298971957110899,0.219508838088322,0.226561512253554,0.23118035309010801,0.241492436019313,0.24722516790385299,0.26161445546306,0.26894335149221599,0.27710727602626101,0.29123186760389103,0.29693280900200297,0.309148964083232,0.318930140069971,0.32219913433881398,0.32764353054471201,0.33630746848281201,0.34473876980950802,0.35343466478142999,0.36104461856874198,0.36430551622629098,0.370816933431448,0.37490039663161701,0.381417729279176,0.38548865326646398,0.39120181884202399,0.39337074349004397,0.39690394613785002,0.39989233613903502,0.40179097443148298,0.40369743242509998,0.40315100045621899,0.402880876634853,0.40287866737362998,0.40124808209638801,0.399348117444567,0.39961985657500199,0.39934753151879399,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.120347295864263,0.122796850865677,0.125495075173474,0.12904017091053499,0.13338800502387199,0.13366713243752501,0.13825722077265801,0.139371464981384,0.14534530732850301,0.14858623334485899,0.150227018145078,0.15484880867645301,0.15647187925498501,0.16028654499316,0.16081603126628399,0.167068972934724,0.16979866266054,0.174410863673854,0.17957488837712701,0.18199916830645599,0.18881774907223101,0.19125825031648,0.195334040296925,0.19613555063699001,0.20103722105518701,0.20646532370873999,0.21217334638101701,0.21842376575939701,0.224668195451354,0.233355006567185,0.237973839377358,0.247196020103194,0.25456065158477698,0.26949269703720602,0.27736728861483601,0.28715869823666801,0.299376681326048,0.30453929539295399,0.31458228187749199,0.32490545927109499,0.32925772394644898,0.332805845628946,0.34418423320295299,0.35261701138365498,0.35805422999882203,0.36702659364586798,0.36892508545687303,0.37299085048811598,0.38087720071312797,0.38657930392981199,0.39010749008982798,0.39500617469449201,0.39663161305526101,0.39799090667277998,0.401248818516796,0.40206271356191797,0.40342569329466499,0.40260899503616498,0.402880876634853,0.40125044185224501,0.40043360112549098,0.399348117444567,0.399348117444567,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.12089003770472501,0.123883807387416,0.12576681430390901,0.130125654591459,0.13338800502387199,0.13448234982882901,0.139072438163962,0.140457685082715,0.14616052471980701,0.14885723605488599,0.15131323824640899,0.15566328563415999,0.15837331674762101,0.16082928683362199,0.162991425176958,0.16897041042736,0.171972575704018,0.177128254978202,0.18229228770785499,0.18498977960843499,0.19153366753576301,0.19315968780911599,0.196963738659126,0.199123208230957,0.20293792212741499,0.21026747029998399,0.21488852842414199,0.222226640744668,0.230918943911332,0.237699150552103,0.242591211386288,0.25398729311603002,0.26108091787439602,0.27682892917810797,0.28416151533568401,0.29557820881046099,0.30616574507766198,0.31268705078355102,0.32082935224862302,0.330340249906172,0.335508472406428,0.34285210085692902,0.35233716360235501,0.35750611048344799,0.36267232237539798,0.37001351080623701,0.37354539913424201,0.377066205037421,0.38685253195382302,0.39146988389680099,0.394453847349159,0.39636340151904098,0.399892482620478,0.402337259918921,0.402608254602567,0.40260619182278801,0.403153954164231,0.40152277493483401,0.40125044185224501,0.39934826793920097,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.12089003770472501,0.12469828835831299,0.12685303440524001,0.13148361382322499,0.134202490007959,0.13557005079735701,0.139887655555267,0.14181564832767199,0.14643152742983401,0.15021593572024999,0.15267046105776799,0.156750246169089,0.15946101370295801,0.161372765094491,0.16462112755234901,0.17059937638234299,0.17387254035583799,0.17903042889124501,0.18501042345899099,0.18716221981109701,0.19289089034712201,0.19506260215575699,0.19995213267350101,0.20129491201321201,0.20728427938674601,0.21379713331400599,0.218423362433778,0.228744697773065,0.23716674267648999,0.243673729319629,0.25101291516854302,0.26078300070407401,0.27058810533757499,0.28579191801277098,0.29366576514361298,0.30345201580940101,0.312955549262873,0.31974784965240999,0.32979750405252001,0.33659026194574199,0.34039830792662901,0.34991290775216699,0.36021760641134398,0.36294163753893199,0.36620345823023398,0.37409034622273701,0.378977976494905,0.38494444259837701,0.39092862292353497,0.39527202246166498,0.39853141918606699,0.400161121561458,0.40179465653352198,0.40233799633932799,0.40260751416896901,0.40206271356191797,0.40206847048330702,0.40016333483587202,0.39962000706963602,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.12116177683516,0.12551424618321499,0.12766973266374099,0.13338652816986701,0.13474596826882901,0.136657007319096,0.14070213652616301,0.14317287113902999,0.14806270665923099,0.149944196589816,0.15375668115909899,0.15810820540085499,0.16054649738388199,0.162458985195823,0.16788200113075599,0.173045032569447,0.17740367621067499,0.18120286909390801,0.186369123124355,0.18797817362281,0.19533506968021899,0.19750973118367601,0.20321005655708699,0.20455431275080399,0.21027267741431099,0.21705727448519599,0.22222624544542999,0.233636002120891,0.240695669270103,0.24883383112945001,0.255633228845912,0.26920546097268799,0.27982576293154199,0.29339694360247698,0.30073321186209101,0.30970350871616797,0.31865986575397098,0.326809384941676,0.33523303110800301,0.342025048567628,0.34800554277755802,0.35562090634530302,0.364564704104274,0.36919164556531198,0.37435268646164699,0.37952514488419398,0.38495846267845502,0.39119667192555102,0.39445975877837203,0.39853215961966498,0.40097707537317101,0.40206403992128997,0.40260913750441901,0.40179525449886699,0.40315099242983898,0.40179097443148298,0.40070755353033899,0.399348117444567,0.39962000706963602,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.12197699422646401,0.12687220541498201,0.12930016744634901,0.13420174556117101,0.135017707399263,0.137200485579965,0.14097461609019599,0.14480183308082301,0.14860544849969301,0.15048767485068501,0.154843637680838,0.15919516593578401,0.16136171477518599,0.164087210717208,0.17032618447704401,0.17413198909118599,0.181206551195947,0.18310357016613599,0.188815515731866,0.18933613285457601,0.197781466300921,0.199137220284654,0.20511002120890801,0.20754123392436399,0.21244585805057201,0.22004493609235301,0.22602912845708201,0.24015626841050999,0.245857980341147,0.253724403070058,0.26514337001710298,0.27735765495168302,0.286073553670319,0.30263165551481402,0.30697805691923802,0.31568327051888001,0.32518161692397601,0.33197095557912099,0.34338153094538898,0.347185886797856,0.35615773274336199,0.36132523086278101,0.36836905995674102,0.374080008244698,0.377612083186049,0.38468597910123198,0.391204784589606,0.39527276690845298,0.39717789051631802,0.40043507396630701,0.40179303319807302,0.40287999774619199,0.40315261576528799,0.40179599091927398,0.40152129406763798,0.400976493460587,0.399348117444567,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.122520472487334,0.127143944545416,0.13011464841724599,0.13474522382203999,0.13474596826882901,0.13747222471039999,0.14151883478466301,0.14480183308082301,0.149963407731459,0.151303632675588,0.155657382231327,0.160010383327089,0.163809588249893,0.16354373245633899,0.171956619259653,0.17657543200387599,0.18283698597855499,0.18473400494874501,0.190445954527665,0.19069335566593401,0.199142379240698,0.20158361690535501,0.20728319783197799,0.21080210750277201,0.216519739759061,0.22466894391133199,0.23145949655652201,0.24368814068575501,0.25129351140982098,0.26377434441327002,0.27111868921822702,0.28523958665424798,0.29422499116295497,0.30969690501163999,0.31458823745179798,0.31948541309693401,0.33170189525316701,0.33849048544833299,0.34963080255136197,0.35343367753663302,0.363222970200618,0.36539837213767201,0.37135820243109402,0.37842931519884998,0.38222870272180998,0.39229099265136702,0.39609462412299801,0.398260424502421,0.40043655483350199,0.401794514065269,0.40233577503853502,0.40260752219535001,0.40315261576528799,0.40152425178883899,0.40043507396630701,0.39962000706963602,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.12360595616825699,0.128772170066802,0.13174508721304501,0.13501696295247501,0.136103927500595,0.138830924375764,0.142605795319593,0.145346051775291,0.15105036425319801,0.152933331037789,0.15701681831709899,0.161098076269236,0.16408058694673,0.166261127773877,0.17549070882250101,0.176849388421914,0.185010162601626,0.18772239896312001,0.193163345832013,0.19340927814265699,0.20077355445690501,0.204571278512513,0.209456374455049,0.214874516370445,0.220050879627088,0.22847183093617501,0.23608054665430001,0.24776422764227601,0.25753835646696699,0.26893370578949199,0.27954410319485301,0.29094612043294898,0.30047499116295501,0.31459043868663999,0.31893533514472699,0.32546147273165699,0.33686568719140603,0.34609770825969099,0.354522859372356,0.35832351707002402,0.36702658561948698,0.371101948195172,0.37516255828356199,0.38331915874543199,0.38847796630140202,0.39554966098174199,0.39772359007798103,0.39989308058582301,0.40125177222480701,0.40314878316861602,0.40287852089218701,0.40151982524001301,0.40179392011311399,0.40016407526947001,0.399348117444567,0.39934826793920097,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.123877695298692,0.12958665103769901,0.13255956818394199,0.13528870208291,0.13691914489189899,0.13964614176706899,0.143147800739647,0.14616200960019299,0.15159310609366,0.154021023979935,0.15810303841843101,0.16137055583326801,0.16408058694673,0.168435044830545,0.17712336089271299,0.17956604330585399,0.18691233651466899,0.19016731471662501,0.193164818672828,0.19694189486468999,0.201589516294997,0.20646902988992,0.21271577117945101,0.21976657319144,0.22331174919230501,0.232273241107011,0.24124360618532101,0.25319827383056398,0.26324415382526101,0.27681340816488398,0.28389120490097203,0.30099827505057403,0.307537262872629,0.320296976478532,0.324641860897048,0.33252522530852602,0.34338376428575301,0.35098753976670199,0.359413435326155,0.36402710115390502,0.37028598635707999,0.37572078501853601,0.381143044467111,0.38902496413010601,0.39255331683751599,0.39935033071898102,0.39989455743982899,0.40261122035014901,0.40206772603651902,0.40342052229905101,0.40315100446940999,0.40043286470508399,0.40070696359137498,0.399348117444567,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.124149434429127,0.13094387384905701,0.13283130731437601,0.13528870208291,0.13827858097767101,0.140460622737965,0.143692015420924,0.147522186119563,0.152679326194992,0.15510872494846301,0.15919073938695799,0.16300025820865999,0.16435232607716399,0.170608957874024,0.17875379968851199,0.181197218522061,0.18772755390597401,0.19233975491928801,0.19479451703502901,0.19802885138642901,0.20403369964128501,0.20837120781615301,0.21543095322257599,0.22275793295339699,0.229019019391414,0.24042617150641399,0.24423052735888101,0.26053007540945,0.26895364131197302,0.28414668659777498,0.29177165974953201,0.30616355186920002,0.31161187698833498,0.32572881340559601,0.33197661618394397,0.33849981611562402,0.350443094326986,0.35696432779545201,0.36457649485717702,0.36973289449900798,0.37463087077559598,0.381424365089226,0.38684957423262201,0.39499954691082301,0.39581418640273403,0.40043876409472501,0.40125325309200299,0.40315469861101899,0.40288073015341003,0.40233503861812703,0.40179230480404599,0.399890122864622,0.39962000706963602,0.399348117444567,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.12469217626958901,0.13067213471862199,0.13337478557524601,0.13583218435697,0.13909379836897501,0.14127584012927,0.14423623411539199,0.148608406220894,0.15485323923847,0.155923946352957,0.160821918616355,0.16490169570129501,0.165710285308931,0.17196839797298599,0.18038423848431101,0.18282618046385399,0.189899994108637,0.19342671144102799,0.19723943680172501,0.20020055516868401,0.20566340201667599,0.211087866713284,0.21841861081654301,0.22819346000887999,0.23554003012782199,0.24314430324435901,0.25129280308174501,0.26976846942382499,0.277644146569413,0.29039153165492099,0.29964990533686903,0.31050990912853099,0.319218363379286,0.33251861759080797,0.33931210789124799,0.34393535119748803,0.35451697202228499,0.363213591375044,0.36892506539092201,0.37299230326298,0.37843596304847099,0.387402637998362,0.39309958627219299,0.397988681358796,0.397988099446212,0.401794510052078,0.40261268917777399,0.40342643774145398,0.402065508748915,0.401519821226823,0.40097856025355699,0.399619120154595,0.39934826793920097,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.125234918110051,0.131215612979492,0.133375526008844,0.13664814218187199,0.13936480107900201,0.14209105752057399,0.144779716389452,0.150240321870699,0.15539598107893199,0.15646668418022899,0.162177668586898,0.165444437541757,0.16733924725072399,0.17278214252347501,0.18310311065585499,0.18554209892738699,0.19207317073170699,0.19587236361494101,0.19859961332109499,0.202917958512603,0.20702209766884999,0.21353057320556601,0.22140626841051,0.23199486215333701,0.24043207892243601,0.24748992007009299,0.259168823355098,0.27547351832214001,0.28525286228753799,0.29555310630555698,0.305080273436309,0.31675918073450399,0.32383866501708503,0.33632076016886198,0.34528743511875398,0.34991362410662402,0.36158072058596402,0.36647446094026198,0.37354094850627501,0.378970584198497,0.38359385559706799,0.39120330773560102,0.39636193670460601,0.39961986058819199,0.40043375162012501,0.40288147058700802,0.40288369188780099,0.40315322175701301,0.40179376961848001,0.40043433754589902,0.39989159971862698,0.39934738102416001,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.12577839637092,0.132303309934829,0.13364726513927899,0.13691914489189899,0.13936480107900201,0.14317727762190499,0.145867413344789,0.15105553926200299,0.15729594573075201,0.158368858093272,0.165166803034871,0.16680313720712101,0.168972639754534,0.17712997663681199,0.184463283162034,0.18852902010094599,0.19343186638388099,0.19695932013667999,0.200233005824905,0.20400491503434201,0.21001197255042101,0.21733418862443599,0.223578708613173,0.237156440817163,0.24287625825553399,0.25237975157710302,0.26758905429653801,0.28307926829268298,0.290417398672565,0.30071911553139902,0.30834262788191202,0.32137359502223201,0.32927050194414997,0.33903815548639998,0.350176534218547,0.356702695884618,0.36429886837667103,0.37055054789678299,0.37761409379435601,0.38440685971395899,0.39038513663628499,0.39528014716529097,0.39907564590691502,0.40124955895039299,0.402335189112761,0.40260973145657297,0.402882951454204,0.40179452209164901,0.40016333483587202,0.399348117444567,0.39934885787816499,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.12713635560268599,0.13311926775973101,0.13364726513927899,0.13719088402233401,0.140722760310769,0.143176537188308,0.14695363344612,0.1518700202329,0.15729594573075201,0.16026881873190199,0.165979807151762,0.16843357600292,0.17087481768076801,0.17848867630217599,0.18473502229246899,0.18988771575312,0.19506230116649001,0.19804627665841901,0.20050474495533999,0.20671936065705901,0.21272789502714401,0.220322586652001,0.22792506185931399,0.23960062015026001,0.24559144029865801,0.25808259121419602,0.27139118884821101,0.288242311770944,0.29639272188687998,0.30561339365318801,0.31540564805156401,0.32898453204115002,0.334700866030399,0.34501643240872598,0.355066373751938,0.35833091739281298,0.36945897419968199,0.37571211853422898,0.38250172005333399,0.39011265707225301,0.39473149389561701,0.39880980616612199,0.39961765132696903,0.40206699362930098,0.40342140921409197,0.40342273958665398,0.401252512658405,0.40125030339718198,0.39989159570543698,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.127679833863556,0.133118531339324,0.13473422166101801,0.13827710412366501,0.141537241281665,0.143991754579612,0.149127550502788,0.152412021639764,0.158655381816524,0.16108404013639699,0.16733924323753399,0.16924805697381701,0.173591476577898,0.18093359606887099,0.18745168520279001,0.190975412708457,0.196149257688229,0.198861494049723,0.20294818786803001,0.21025345824628699,0.21626051174917599,0.22195376186820701,0.23118519500412399,0.243131019584689,0.25156897678738699,0.26677383289204298,0.28063328101738499,0.29340314598798201,0.30264347436004901,0.31132066786548701,0.32328756771455902,0.33523011753189402,0.34067765405914902,0.35099101518944298,0.359414203852085,0.366208418533362,0.37408298603185097,0.38033021091080499,0.38766181784996401,0.39418801162155698,0.39744888519996402,0.40016702897748102,0.40124882654317601,0.40233651948532301,0.40369314834452702,0.402879261325785,0.40070829396393698,0.40016555613666499,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.128766053964887,0.13393448916422601,0.135005960791452,0.13909158509456199,0.14235172225256201,0.14453449642007399,0.149127550502788,0.153227239031068,0.16001334506147999,0.16162751839726699,0.167882725511593,0.171148025638828,0.17522043851969099,0.184192996806464,0.187996644330855,0.192063105650604,0.19832317073170699,0.199676711441027,0.20593733034238301,0.21297011714341799,0.21897495737189299,0.22575737728707701,0.23634602922116199,0.24639337403029299,0.25591533404671801,0.27329557603566801,0.284977425002304,0.29829224107458502,0.308618793561173,0.316482246529313,0.32790345082991201,0.34147792031024199,0.34637975727583398,0.35723955438819899,0.36348881796779098,0.370556252646699,0.37707360134701901,0.384948303287381,0.39201113283049699,0.39609018954779102,0.39772062433039901,0.40179746376009001,0.403150264035812,0.40260752219535001,0.40315040650406497,0.40097708339955102,0.39961985657500199,0.39934885787816499,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.129580534935784,0.134206228294661,0.13473422166101801,0.139363324224997,0.142894460079834,0.145621452941813,0.14994203147368501,0.15431419956599801,0.16109956516281199,0.16217173307854399,0.16924068474335999,0.17332120627508901,0.17793488815559899,0.18527995734139299,0.19071183038716999,0.19369207160558799,0.19940939083303899,0.20456359726640699,0.208923511082345,0.215143297779678,0.221688670587393,0.23064943410807201,0.24123659714858001,0.25019403574115101,0.26270586261276702,0.27954337480082497,0.29122152962585202,0.30263859432072598,0.315139071890363,0.32300031158408998,0.334695472302727,0.34582354114916602,0.35290075998586101,0.36294682057411698,0.36837718466036701,0.37327365197742701,0.38223443556405701,0.387665694591729,0.39363935835188202,0.39826115289644798,0.39989380496665999,0.40206994332412199,0.40342200316624699,0.40233504263131697,0.402335189112761,0.40043507797949701,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.130666755037116,0.13474970655553001,0.135549439052322,0.13963432693502401,0.14370894105073001,0.14697941217357899,0.150755776024174,0.15512941695730201,0.16164378385727901,0.164619606553251,0.16978269016341399,0.174133473971572,0.17956385009739201,0.18663939744035499,0.19261326787980601,0.19586303896743501,0.20103908919523999,0.20809473312124399,0.21245538737077899,0.216227304606597,0.224948807745393,0.23526827093143601,0.24449746671379799,0.25426864584366698,0.27329706492924399,0.284977437041874,0.296925846116949,0.30752916224814397,0.32057312610503202,0.32843289295794298,0.33876861759080801,0.34962863342204098,0.35860433604336001,0.36674969957257902,0.37245327964326902,0.37843080409242702,0.38549457272205701,0.39011134676564202,0.394999534871252,0.39962280626982299,0.40125176821161601,0.40315542700504597,0.40260678577494202,0.40151982524001301,0.40125044185224501,0.399619120154595,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13093849416754999,0.13474897013512299,0.136635659153653,0.13990606606545899,0.143980680181165,0.147795369998481,0.15184347297951101,0.15512941695730201,0.16300247950945301,0.165976829364609,0.17005442929384901,0.17522043049331101,0.18037833106828899,0.18718213928081701,0.19288574744383899,0.198308695154538,0.20294052668787599,0.21026717332390699,0.213540871051703,0.218401221663265,0.22657924654119199,0.23798787952338701,0.24965682809001999,0.26078597046484597,0.27845644235822797,0.28932527516840101,0.30100194511304201,0.31540665134912199,0.32709192356702599,0.33440822419863803,0.34202875474880801,0.35533295392632902,0.36267895015906698,0.37082505412188299,0.37544167767083397,0.382505426234513,0.38820975476518199,0.393372216330859,0.396629977680241,0.39989306854625201,0.40206624918251299,0.40288221102060601,0.401791564370448,0.40097634697914297,0.40070696359137498,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13148197242842,0.13447796742509499,0.13717840099411499,0.140177068775486,0.144251682891192,0.148610587389786,0.153199222950054,0.15730186117315501,0.16544739927614899,0.167335525016783,0.171955870799675,0.17848204450531699,0.18200803344368,0.188269103828937,0.193700228414735,0.20021086906758201,0.20457096147048401,0.211083127135619,0.216258262356051,0.22084540099636299,0.23092559978733301,0.242876982636371,0.25454665959703099,0.26730698521444402,0.28334775473243401,0.29285567460283102,0.30724679418337802,0.31921099917520901,0.32899484192685802,0.338212584064296,0.34746428983067201,0.36022279747290997,0.36593908330387598,0.37462792910715498,0.38114673058234,0.388209010318394,0.39201189333004599,0.39581713208436398,0.39989233613903502,0.40206698560292098,0.40260972744338303,0.40288221102060601,0.400977819819959,0.40043360513868098,0.39962000706963602,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13202471426888199,0.134750446989128,0.13799361838542001,0.14044880790592101,0.14533716255892601,0.149154065650655,0.15428618348498399,0.15757360030358999,0.16707783405875801,0.16815000197449001,0.173586305582283,0.179296525476213,0.18445590691838701,0.18962706306070301,0.19451470938563201,0.20048187177760901,0.20647313538352799,0.21298456462825499,0.21788944158544801,0.223562800327091,0.234458948916583,0.24586463621714799,0.25699157535053602,0.27437295909210702,0.28851006580347799,0.29774624253024901,0.31376707652575903,0.324101567102627,0.335244857979619,0.341471980788698,0.35126790524954199,0.363213408774889,0.36947169199952901,0.379245289076514,0.38603730252294799,0.39282858757535599,0.39418433353270899,0.39690187934488003,0.400435814399904,0.40206698560292098,0.40260899102297498,0.40206625319570299,0.40043360112549098,0.40016186600824699,0.39934826793920097,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13256819252975099,0.13475118340953601,0.13826609794945199,0.141264765730823,0.14615237995023,0.14996854662155201,0.156188357398027,0.159201825824975,0.16843579329052399,0.16896521936579401,0.17494426481405001,0.18092696025882199,0.18717404266952301,0.190714019582442,0.195874145471404,0.20183909458896701,0.20837457287616401,0.21434252386002101,0.22060756931020301,0.22709467661552599,0.23961536059798499,0.24939430725754999,0.26296836337928597,0.27926131775830298,0.29258394349877598,0.30317734303690602,0.31783948138024198,0.327903705667491,0.34067670293306401,0.34527560022075798,0.35914318910248699,0.36701407048574702,0.374358577824909,0.38277495209053503,0.38902496413010601,0.39473076550158898,0.39744520309792603,0.397988099446212,0.40152055764722999,0.40206698560292098,0.401794510052078,0.401523511355242,0.39961985657500199,0.39934738102416001,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.133111670790621,0.13448018069950901,0.13962184390680499,0.14235172225256201,0.146695121790692,0.15132576943291101,0.157274577499359,0.160831528200367,0.16979523338948599,0.17059491772799501,0.177931926421207,0.182012443939746,0.18961674916180499,0.190985022292469,0.19696110199314301,0.204013748066044,0.21027674678920699,0.216516436903499,0.222238008106002,0.230625084076335,0.243692196014485,0.25374066451688099,0.269215417697655,0.28659977119999802,0.29720203988854199,0.30996714722211799,0.32354011175611103,0.334423235536703,0.34284914313572701,0.34880453082756202,0.36294681254773697,0.369458986239252,0.37951867562153901,0.38685105108662698,0.39391554811028501,0.39608946115376298,0.39826189734323603,0.39961853422882099,0.40206551677529601,0.40206624918251299,0.40152203048804602,0.39961985657500199,0.39961985657500199,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13365514905149101,0.13556492796002501,0.13989358303724,0.14316842051106199,0.14778060547161601,0.151597508563345,0.15836301488829399,0.16218948743213299,0.17142419533127901,0.171952880972951,0.180376842174713,0.184459580994045,0.19097618926076701,0.19207124239380099,0.199134282629404,0.20673040294998399,0.21163470602097301,0.21787513255567301,0.22522419285915399,0.23469970220523201,0.246952337185675,0.25727327321253302,0.27627916224814397,0.28958816922756397,0.30263462126239499,0.31512798143915599,0.327889422723454,0.33985433604336002,0.34746208056944899,0.35614150340206202,0.36539468602244402,0.37163290329592102,0.38277880876634901,0.39011045182421999,0.39554524647248601,0.398806116037703,0.39989085928502899,0.400976493460587,0.40233578306491502,0.402065512762105,0.40152203048804602,0.399348117444567,0.399348117444567,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.133926888181925,0.13583593067005201,0.140436324877702,0.143711898771932,0.14778060547161601,0.15241198953424201,0.15890575672875601,0.16354818308430699,0.17251115185301799,0.17249635923382101,0.18173480541966899,0.18636175892027901,0.192334148492533,0.193158939349138,0.19886254349896901,0.20863110402221199,0.213536143513609,0.220047572758336,0.22875311945276799,0.23877653762173201,0.24994073922643001,0.26270806786079898,0.28198494756686698,0.294205533210112,0.30752593764979202,0.32110698675550903,0.33413943877621499,0.34338473547779003,0.35181065110319398,0.36130603978708897,0.36810913164516101,0.37679447794655702,0.38658242017202799,0.39391185396867601,0.39663146657381798,0.398806116037703,0.40124955895039299,0.40179097443148298,0.401520565673611,0.40097707938636101,0.40016333483587202,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.134470366442795,0.13692362762538901,0.140436324877702,0.144255377032801,0.148865352732132,0.15376921635879101,0.15944997542322301,0.16490614231607301,0.17332710566473,0.17521375856454899,0.18282176595459801,0.188261723572099,0.19314862946343,0.19505890400095799,0.20103498771482201,0.20998906325397801,0.215710056557087,0.22140626841051,0.233097263437686,0.24393884869277499,0.254288565313387,0.26841607448031601,0.28687477907387798,0.29828532634781402,0.31159982337147202,0.32681352053420998,0.33930175386044797,0.34827530340520801,0.35697296618742802,0.364567653799095,0.37245622532490003,0.38032856750940403,0.39011429244727203,0.395813291461312,0.39744668396512201,0.39989307657263301,0.40206329948769198,0.40179097443148298,0.40125178025118702,0.40043360112549098,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13555732296453399,0.137195366755824,0.140436324877702,0.14371115833833401,0.15104001022239799,0.15485470003971399,0.15972097813325001,0.16653436783745901,0.173871324359198,0.17684346093994099,0.18363771976630999,0.18989290280149601,0.193691375317082,0.19641759965313199,0.20347990346832701,0.21216593401865799,0.21761149404972299,0.22222074938140701,0.23635445491405499,0.24801198996766599,0.25754722561738103,0.27466461367907102,0.29013564863909502,0.30235994046351999,0.31920484093479701,0.32952870257733502,0.343919117842997,0.35153470012961002,0.361589585723188,0.36945527203169198,0.37707506214826397,0.38521840302960603,0.39201646636031601,0.39744372624391999,0.39962060102178998,0.40179451807845901,0.40206403590810003,0.40151923530104899,0.40098004112075197,0.39961985657500199,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13610080122540399,0.13746710588625899,0.14179428410946801,0.14452711616323599,0.151854491193295,0.15539892274737199,0.16135067649545101,0.16789232706922499,0.17468727817091001,0.177928944620864,0.184723203447234,0.19043638106236499,0.19450659270838699,0.19777556289808801,0.206196562365458,0.21325289455358701,0.21978393425238599,0.22656783904795599,0.23906963695718,0.25045764615477001,0.26624068458283201,0.28036966659057699,0.29502548014610602,0.307246093881683,0.32762730521660199,0.33441780569031798,0.34853426453794201,0.35615426534700101,0.366476475561758,0.37380089287061602,0.38223664081209002,0.38929450603888799,0.39446211853422902,0.39798720450479003,0.40016407928266001,0.403424952861067,0.40206403590810003,0.40070401790974403,0.400164819716258,0.39907637831413301,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13691601861670799,0.13773884501669301,0.14260876508036499,0.14561481311857299,0.15294144771503401,0.155940928167426,0.16243689659678301,0.16897707432974099,0.17523149686537701,0.17874490244576699,0.186897120503903,0.190979859323235,0.19668050976505499,0.19859004386898499,0.207555258017632,0.215423861915435,0.22141363261458699,0.22955549664192301,0.242058771405153,0.253989526456395,0.27004282716088601,0.28607324666126699,0.30045805349357801,0.311051922574966,0.33115770866422101,0.339306904790112,0.35152266256550702,0.36104409685401201,0.37028157184782401,0.37651828818815403,0.38630978208698102,0.39092420038789899,0.39636355602686502,0.39907416503971899,0.40124956296358399,0.40288073416659997,0.40206403590810003,0.40016053964887499,0.400164819716258,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.137459496877578,0.138554062407998,0.14260876508036499,0.14643077094347501,0.15375666510633801,0.15702641184835001,0.16433907452301599,0.16924807302657799,0.175503235995812,0.18064782481878799,0.18798482147243001,0.192882773669876,0.197765993445979,0.20157917831695801,0.209727702233485,0.217867304828125,0.22249985271591799,0.234173589018499,0.244503687158658,0.26105402348005202,0.27765005799862502,0.29014565151575,0.30480587958053501,0.316755510672036,0.334417845822221,0.34311051619579103,0.35532775885157297,0.36539118652056102,0.37272427834010602,0.38168208413958299,0.38930113382255699,0.39391407526946998,0.39772151525863098,0.39961764330058902,0.40206478035488802,0.402065512762105,0.40206403590810003,0.39988880051844,0.399893817006231,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.137459496877578,0.139369279799302,0.14369572561529401,0.14670251007391,0.154571882497643,0.15865611422374101,0.16596803646481001,0.17060529583793599,0.17659019653074101,0.18064782481878799,0.18961673712223401,0.192884250523882,0.19885294996771799,0.20375531667442001,0.21054218320438201,0.220041221884793,0.223857811947685,0.237706197714151,0.24885078083839701,0.26404315792802502,0.28172540853473899,0.29612171516366298,0.31051240131966501,0.32218808803269899,0.33795045853106298,0.34745466820708998,0.35804294490788802,0.367293360433604,0.37680037332300798,0.38439875106309401,0.39038661750348103,0.39554451005207802,0.399349740780017,0.40097634296595303,0.40206478035488802,0.40152203450123602,0.40152129406763798,0.39988953693884799,0.39934885787816499,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.137459496877578,0.140185978057802,0.14451020658619099,0.148061209739274,0.15511536075851201,0.15947281248224199,0.16732673211698401,0.171419776808833,0.17740467750163799,0.18200578806374501,0.190705170497979,0.19424368660965399,0.19993991050264701,0.206200968848333,0.21298857982508301,0.22112891884013,0.226031724991163,0.24150980911983,0.252653663850049,0.268930784187003,0.28634497776532097,0.30019559687215203,0.31458775185577897,0.32680618843565501,0.34420121100423201,0.35180249429404598,0.36130307805269701,0.37082523270884898,0.380059033627003,0.38820311092875198,0.39255832128573598,0.396087988312948,0.40043669730175602,0.40124955895039299,0.40152203851442603,0.40125103179120902,0.40124955493720299,0.39961853422882099,0.39934885787816499,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.137459496877578,0.14072945631867201,0.14532542397749501,0.14887716756417699,0.15593058216300701,0.160559769003981,0.16895643449237499,0.17223425777973,0.17876264074659501,0.18363696729314199,0.19124791233844099,0.19533211998539801,0.20265730983337499,0.20756040493410499,0.214892234605322,0.223302839909989,0.22956359726640699,0.24558589607635201,0.256998548268565,0.27545032208259501,0.29177829555958201,0.30562891867960301,0.321376811594203,0.33196702666588301,0.346102648496868,0.35587858927694799,0.36592191888925202,0.37327014846235401,0.38331769393099702,0.39037628755182202,0.39418655082031201,0.39690246928384498,0.40043669730175602,0.401248818516796,0.40070755754352899,0.40097855222717599,0.400977815806768,0.39934753151879399,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13827471426888199,0.14154393728956799,0.146683383209262,0.14996560495311201,0.156473324003469,0.16083150813441599,0.16976943860926599,0.17250673333057201,0.17985033770193201,0.18418118598760899,0.19314935384426701,0.195332856405806,0.204013792211136,0.210276331424018,0.21598066798106699,0.22330357633039699,0.23255125486037501,0.24938950748203101,0.26161516780432598,0.282513338239057,0.296126870106517,0.30997453550533599,0.32517968657947399,0.33658733633006299,0.34881857097358998,0.359412678839796,0.36999875029256202,0.37788676799811499,0.38576335011809998,0.39173572363759401,0.39717642168869299,0.398260428515611,0.40206639566395702,0.401248818516796,0.40016333884906202,0.40016333483587202,0.39989159570543698,0.39934753151879399,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.13936167079062101,0.142087415550438,0.146954385919289,0.15050908321398099,0.1575595441048,0.16218873094577399,0.17058465600056999,0.172778472461007,0.18039381997599099,0.185266669668533,0.19478052906047399,0.19669229650476799,0.20537175144290201,0.21217850533706101,0.217068368949594,0.22411879372170099,0.235267909744315,0.25265037704724902,0.26487457255510799,0.28713291549601899,0.301017450073506,0.31513685058956997,0.33007099092730102,0.34093369358939402,0.35180696900115499,0.36375977251953501,0.37217340376963798,0.38087442559208201,0.38820900229201299,0.392549464174893,0.398806124064084,0.39880317035607299,0.40206639566395702,0.400977815806768,0.39961985657500199,0.39989159570543698,0.39961985657500199,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14017615176151799,0.142630893811308,0.148042082874626,0.150780822344416,0.159462458451442,0.16273147278623601,0.17139987740506499,0.17386543299593599,0.180395296829997,0.18581014792940301,0.19586674916180499,0.198050992156942,0.20645944839823899,0.213807467278855,0.21951624242430101,0.22574922850431001,0.23934178743961401,0.256724991162955,0.26894771382999899,0.292293753726247,0.30672325144498902,0.32193107329722698,0.33251590668080599,0.34310761064606299,0.35724102321582402,0.367020658137514,0.37543428136123602,0.38359034405561399,0.39092566118914401,0.39445385938873001,0.39934960232495298,0.40016187002143699,0.40260840108401102,0.40070607667633401,0.39961985657500199,0.39961985657500199,0.39961985657500199,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14017615176151799,0.14344537478220401,0.14831382601825099,0.15295399896748599,0.160550155406779,0.16408795917718699,0.17275931349083701,0.17468065038724101,0.18093951552446499,0.18825506769609801,0.196954446117142,0.20022343637279499,0.20836014947046799,0.214350945539724,0.221417679916937,0.22928258164675,0.243417137975728,0.26052786614822698,0.27302602208731502,0.29799881065094302,0.30916817121168499,0.32546220915206397,0.33767747731825098,0.34691196649853001,0.36158886134235102,0.37109601268681802,0.38005238577738198,0.38685121362083202,0.39282710269497001,0.39608503460493699,0.39934886189135599,0.40016187002143699,0.40206492282314099,0.40043507396630701,0.39961985657500199,0.39907637831413301,0.39961985657500199,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.140990632732414,0.144261336620297,0.14885656785871301,0.15376922037198101,0.162722595609441,0.164631437438056,0.173302051318108,0.176040086473013,0.18175473291576899,0.18988477007148899,0.19804066621847399,0.20049517951642001,0.21053406652713599,0.21489368738018599,0.223049595566741,0.23308620107881001,0.24613379285966799,0.26595970307529199,0.27954926215089698,0.30098499540409501,0.31242388583404801,0.32981078369899902,0.340394132202192,0.34963009823647601,0.36566495231206297,0.37381340800435597,0.38249951881849098,0.38983960763520698,0.39554523844610601,0.39690099242983901,0.40016334286225202,0.40043286871827399,0.40043522446093999,0.39989159570543698,0.39961985657500199,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14099136915282201,0.14507581759119301,0.14967104882960899,0.15539891873418199,0.16380955213118101,0.16544518198854499,0.17357379044854301,0.177124833733529,0.18311269214753501,0.192331166692191,0.19858340805893501,0.201311137341322,0.21134928391843999,0.21625238303236,0.22413507924766499,0.23662029866803799,0.24993519500412401,0.26976257806056297,0.28606879603329899,0.30560235537345298,0.31731519820825499,0.33388466540748801,0.343654265347001,0.35479241332070899,0.36973883000736202,0.37652933048107901,0.38494591945238299,0.39119756686697299,0.39635971941700199,0.39825821524119798,0.40125029938399098,0.40016260242865398,0.40043522446093999,0.39989159570543698,0.39961985657500199,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14071963002238699,0.14561929585206301,0.15048700665451201,0.155942396995052,0.16435303039204999,0.16680387764071899,0.173845529578978,0.177668311994398,0.18311342856794299,0.194776082445696,0.200487062839175,0.20239809386306201,0.21243624445337,0.218426300089029,0.22549377489983899,0.239066691275549,0.252922852598091,0.27410966772711198,0.29340281489978698,0.30804875600734399,0.32274851198932603,0.33741506885510703,0.34691513491221898,0.35914098385445398,0.37272648760132898,0.37897498666818202,0.38847705530721899,0.39309900435960898,0.39717420038789902,0.39853069079204001,0.40152203851442603,0.40016260242865398,0.400163485330505,0.39961985657500199,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14153484741369099,0.14589029856208999,0.15048700665451201,0.15784383448768699,0.16489577223251201,0.168704582726138,0.17438827141943999,0.17875526851613699,0.18420186194368801,0.194776082445696,0.20184649892494699,0.20457201091973001,0.214608688669223,0.21978278647998001,0.22712200042122399,0.24205434886951599,0.25781342052550998,0.276555319901025,0.29829412326080401,0.314300977308138,0.32818256219080399,0.34203095598365102,0.350173795216213,0.362131591143242,0.37435692238393797,0.38277639683901898,0.39200671832124101,0.39500044185224498,0.39771841908236599,0.39907490547331698,0.40152203851442603,0.39989086329821999,0.39989174620007101,0.39961985657500199,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14153484741369099,0.14589029856208999,0.15211818187071799,0.15893153144302399,0.165710993637007,0.16924584771259399,0.17520422924434201,0.17957123035423,0.185832296726296,0.19668121809313099,0.202660979895843,0.20538722831103401,0.215965911480581,0.222229919521088,0.23065166744843599,0.24531596288152199,0.26080255096029198,0.283071904088606,0.30182452269523302,0.31973355466880099,0.33415787737873798,0.34474761488078098,0.35452014846235402,0.365662731011269,0.376530839440606,0.38386409780754599,0.39282193972573498,0.39635840108401099,0.39853363647367102,0.40043434155908902,0.40043508199268701,0.39989159971862698,0.39934826793920097,0.39961985657500199,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.142078325674561,0.14643451324336701,0.15293339926202201,0.160290227095198,0.16679795417193599,0.16951758684302901,0.174660750983472,0.17984223306425701,0.18691777639403001,0.19695295722356601,0.204018202707202,0.20674371470198499,0.217324607132755,0.22440384460413701,0.234455290893686,0.25102323709382102,0.26270398845292797,0.28687477907387798,0.306444832359412,0.32244873671192598,0.33741875497033602,0.34773601290834599,0.358594762578061,0.37028009098062797,0.37951776462735598,0.38630975800784001,0.39445311494194202,0.39635840108401099,0.39934959429857297,0.40070534025592602,0.400435818413095,0.399348121457758,0.39934826793920097,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14235006480499601,0.14643451324336701,0.153205138392457,0.16110397164568699,0.16734217286640299,0.17006106510389801,0.17520422924434201,0.18147119500605,0.18936269214753501,0.19831312972974499,0.20618917006904899,0.20918715761467499,0.21895578234896201,0.22494732286500699,0.23852917260217499,0.25373915957054399,0.26650686343820001,0.29122334158124202,0.30861801299567299,0.32788574463460501,0.33877819506929802,0.3523548537449,0.36239763756333199,0.37327143870301399,0.38169094526361702,0.3892996369026,0.396084290158149,0.39717361847531502,0.40016407928266001,0.40070534025592602,0.40016407928266001,0.39961986058819199,0.39934826793920097,0.399348117444567,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14316528219630001,0.146706252373802,0.15374861665332701,0.16164671348614901,0.16815739025770801,0.17087554607479499,0.17602018706924399,0.182285675976947,0.18963369485756201,0.199942828091946,0.207276867024386,0.21190308009139799,0.22058400787034799,0.22603354296633801,0.239616129123914,0.25509712281550001,0.269767733003417,0.296114645929068,0.31242016360010799,0.33250310861715399,0.34149485396642898,0.35669973816341599,0.36728820549075097,0.37653231629461198,0.385765571418894,0.39065907298837199,0.39581255102771401,0.39826057499705397,0.40070829797712698,0.400706813096741,0.40016407928266001,0.39961986058819199,0.39934826793920097,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14316528219630001,0.14697799150423699,0.15483483675465801,0.162732933587481,0.168971134808197,0.17114728520523001,0.17737814630101001,0.18418638106236501,0.190449652682464,0.20102978461368501,0.207820345285256,0.213532778453599,0.221398488841244,0.228206723602599,0.242332047587447,0.25808257114824401,0.274386561800401,0.29937551549428498,0.317581001830336,0.33440454610978998,0.34611369078979298,0.35886996910485602,0.37027659950512498,0.38060619800310103,0.387938011621557,0.39256125091460597,0.39771251567953397,0.39853231412748902,0.40098077754116002,0.40097855222717599,0.39962133744219802,0.39934885787816499,0.39934826793920097,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.14316528219630001,0.14779099562112799,0.15565079457956099,0.16327641184835001,0.169513876648659,0.172234245740159,0.17819262727190699,0.185002338887268,0.19262282930553501,0.20211674113542499,0.20863482625615301,0.214620475408936,0.22302745078303801,0.23200812574705501,0.24613713584713101,0.26161592429068498,0.28172130906091702,0.30399287145045401,0.32165561995923198,0.33712415871493201,0.349915096947439,0.36240110897288302,0.37299399080947299,0.38250837994252501,0.39038293138825197,0.39473295469686098,0.39771251567953397,0.39798883586661998,0.40097930068715398,0.400706813096741,0.39934885787816499,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598],[0.143708024036762,0.14860474017161701,0.157009498258115,0.16409089281924699,0.16951313621506101,0.172234245740159,0.17846436640234201,0.18554655758173499,0.19371052626087201,0.20320148839594099,0.20999204906751101,0.21625091019154499,0.22547384740373899,0.23418203879053401,0.24967122540997999,0.26596302198361399,0.285253917756569,0.30535156710262801,0.32654693233343901,0.33848138152628998,0.35290128170059099,0.365662718971699,0.37570990927300602,0.384681556565595,0.392556848444921,0.39663440021587798,0.39852847350443699,0.39934753151879299,0.40070829797712698,0.40043507396630701,0.39934885787816499,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598,0.39907652880876598,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907637831413301,0.39907652880876598]],"type":"surface","opacity":0.80000000000000004,"contours":{"z":{"show":true,"usecolormap":true,"colorscale":"Jet","highlightcolor":"#1EE4F4"}},"frame":null},{"x":[1],"y":[0.0001],"z":[0.069007747142688897],"type":"scatter3d","mode":"markers","showlegend":false,"marker":{"color":"red","line":{"color":"rgba(255,127,14,1)"}},"error_y":{"color":"rgba(255,127,14,1)"},"error_x":{"color":"rgba(255,127,14,1)"},"line":{"color":"rgba(255,127,14,1)"},"frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.20000000000000001,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

]

- Accuracy as a function of penalty parameter on CV folds.

- Performance on test data:

| Not_penalized| Penalized|
|-------------:|---------:|
|     0.9336957| 0.9369565|

]

---
## III. Logistic Regression<h0br> 
### &nbsp; Confusion matrix <h0br>
.pull-left[ <h0br>
- .red[Imbalanced data] `$=$` nightmare!

]

---
count:false
## III. Logistic Regression<h0br> 
### &nbsp; Confusion matrix <h0br>
.pull-left[ <h0br>
- .red[Imbalanced data] `$=$` nightmare!

- .red[Accuracy] `$\approx$` right for .red[wrong reasons]!

]

---
count:false
## III. Logistic Regression<h0br> 
### &nbsp; Confusion matrix <h0br>
.pull-left[ <h0br>
- .red[Imbalanced data] `$=$` nightmare!

- .red[Accuracy] `$\approx$` right for .red[wrong reasons]!

- .stress[Confusion matrix]:

- .stress[Recall/TPR] `$\displaystyle=\frac{TP}{P}=\frac{TP}{TP+FN}.$`

- .stress[Precision] `$\displaystyle=\frac{TP}{TP+FP}.$`

]

.pull-right[ <h0br>
- .stress[F1 score] `$\displaystyle=\frac{2\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}.$`

- Confusion matrix without penalization:

|        | nonspam| spam|
|:-------|-------:|----:|
|nonspam |     549|   27|
|spam    |      34|  310|

- Confusion matrix with penalization:

|        | nonspam| spam|
|:-------|-------:|----:|
|nonspam |     551|   25|
|spam    |      33|  311|
]

---
## III. Logistic Regression<h0br> 
### &nbsp; Receiver operating characteristic (ROC) curves <h0br>
.pull-left[ <h0br>
- .stress[TPR (power)] `$\displaystyle=\frac{TP}{P}=\frac{TP}{TP+FN}.$`

- .stress[FPR (error I)] `$\displaystyle=\frac{FP}{N}=\frac{FP}{FP+TN}.$`

- .stress[ROC curve] `$=(\text{FPR}(\delta), \text{TPR}(\delta))$` for `$\delta\in (0,1)$` s.t `$$\hat{y}_i=1\Leftrightarrow\hat{\mathbb{P}}(Y=1|X=x_i)\geq\delta.$$`

- Is `$\delta=1/2$` always good?

- How much you want to control .red[FPR]?

]
.pull-right[ <h0br>

![](itc_linear_files/figure-html/roc-1.png)

]

<h0br>

- AUC = .stress[Area Under ROC Curve].

- Which part is for fine-tuning `$\delta$`?

---
## III. Logistic Regression<h0br> 
### &nbsp; Multinomial case ( `$M$` ) <h0br>

- Multiple classes: `$Y\in\{y_1,y_2,...,y_M\}$`.

- Example: Hand-written digit recognition i.e., `$Y\in\{0,1,...,9\}$`.

- `$\text{Softmax}: \mathbb{R}^M\to{\cal S}_M=\{(p_1,...,p_M)\in[0,1]^M:\sum p_m=1\}$` defined for any `$z=(z_1,...,z_M)\in\mathbb{R}^M$` by:

`$$\text{softmax}(z)=\left(\frac{e^{z_1}}{\sum_m e^{z_m}}, \ldots,\frac{e^{z_M}}{\sum_m e^{z_m}}\right).$$`
--

.pull-left[<h0br>
- Recall binary model: <br>
.left[<img src="./img/logR0.png" width="300px" height ="200px"/>]
]
.pull-right[<h0br>
- Multi-class model: <br>
.left[<img src="./img/logRM.png" width="330px" height ="190px"/>]
]

---
## III. Logistic Regression<h0br> 
### &nbsp; Multinomial case ( `$M$` ) <h0br>

.pull-left-40[
- Multi-class model:
.left[<img src="./img/logRM.png" width="300px" height ="150px"/>]
]

.pull-right-60[
- For any training data `$x_i\in\{1\}\times\mathbb{R}^d$`,
`$$\hspace{1.1cm}\hat{p}^i=\text{softmax}(x_i^tW)\hspace{5.5cm}$$`
.center[<img src="./img/matrix_form_logR.png" width="400px" height ="125px"/>]
]

- .stress[Cross-entropy:]

`$${\cal L}(W)=-\sum_{i=1}^{n_{\text{train}}}\sum_{m=1}^Mt_{im}\log\left(\frac{e^{x_i^tw^m}}{\sum_{j=1}^Me^{x_i^tw^j}}\right),$$` where `$t_{i}=(0,...,0,\underbrace{1}_{m\text{-th}},0,...,0)\Leftrightarrow y_i=y_m$` (one-hot encoding).

- .stress[Optimization:] `$W^*=\arg\min_{W}{\cal L}(W)+\lambda{\cal R}(W)$`.

---

## III. Logistic Regression<h0br> 
### &nbsp; Mnist dataset (Tensorflow)<h0br>
.pull-left[ <h0br>

- Dimension: 
  - Train: `$60\ 000\times 28\times28$`.
  - Test: `$10\ 000\times 28\times28$`.
  
- Examples:

]

- Model summary:

```r
summary(model1)
```
.center[<img src="./img/summary_mod1.png" width="370px"/>]

]

---
count:false
## III. Logistic Regression<h0br> 
### &nbsp; Mnist dataset (Tensorflow)<h0br>
.pull-left[ <h0br>

- Dimension: 
  - Train: `$60\ 000\times 28\times28$`.
  - Test: `$10\ 000\times 28\times28$`.
  
- Examples:

]

- Model summary:

```r
summary(model1)
```
.center[<img src="./img/summary_mod1.png" width="370px"/>] <h0br>

]

---
## III. Logistic Regression<h0br> 
### &nbsp; Mnist dataset (Tensorflow)<h0br>

|         | Multi_Logistic_Regression|
|:--------|-------------------------:|
|loss     |                 0.2689023|
|accuracy |                 0.9272000|

]

- When it works:

- And when it doesn't:

]

---
template: inter-slide
class: left, middle
count: false

<br>

<br>

<br>

<br>

---
## IV. Deep Neural Networks (DNN)<h0br> 
###  <img src="./img/neural.png" width="32px"/> Key to a bigger world<h0br>

.pull-left[ <h0br>
- .stress[Synapse] links between .stress[neurons]:
.center[<img src="./img/brain.png" width="280px" height = "175px"/> <hbr>
.caption[[Source: Perceptron: An introduction to artificial neural networks](https://matt.might.net/articles/hello-perceptron/)]]
]

.pull-right[ <h0br>
- Artificial model:
.center[<img src="./img/NN0.png" width="270px" height = "180px"/> <hbr> 
.caption[[Source: Neural Networks from scratch](https://github.com/skawy/Neural-Network-from-scratch)]]
]

- Governing equation:

`$$\begin{aligned}
z_0 &= x\\
z_{\ell} &= f_{\ell}(z_{\ell-1}^tW^{\ell-1})\\
\hat{y} &= O(z_{L}^tW^L),\\
\text{for }\ell &= 1,...,L,
\end{aligned}$$`

]

.pull-left-60[ <hbr>
 
  - `$L$` : depth.
  - `$W^{\ell}$` : weights.
  - `$f_\ell$` & `$O$` : activation functions,
    - `$f(x)=\max\{x,0\}$`: `Relu`
    - `$f(x)=\tanh(x)$`: `tangent hyperbolic`
    - `$f(x)=\sigma(x)$`: `sigmoid`
    - `$O(x)=\text{softmax}(x)$` ...

]

---
## IV. Deep Neural Networks (DNN)<h0br> 
###  <img src="./img/neural.png" width="32px"/> Feedforward NN<h0br>

.pull-right[ <h0br>
- [Universal approximation theorems](https://en.wikipedia.org/wiki/Universal_approximation_theorem):

]

---
count:false
## IV. Deep Neural Networks (DNN)<h0br> 
###  <img src="./img/neural.png" width="32px"/> Feedforward NN<h0br>

- Training:
  - Loss: `$\sum_{B}\ell(f(x,W),y)/|B|$`
  - Metric: `$\sum_{V}\ell'(f(x,W),y)/|V|$`
  - Opt: SGD, Momentum, Adam, ...
  
  `$$\text{Opt. + Loss}_t\rightarrow W_t\rightarrow \text{Metric}_t\rightarrow \text{Stop}.$$`

]

.pull-right[ <h0br>
- [Universal approximation theorems](https://en.wikipedia.org/wiki/Universal_approximation_theorem):

- Backpropagation:

]

---
## IV. Deep Neural Networks (DNN)<h0br> 
### <img src="./img/neural.png" width="32px"/> Challenges in DNN <h0br>

.center[<img src="./img/cost_dnn.jpg" width="400px" height ="250px"/> <hbr>
.caption[Source: [https://reconsider.news/2018/05/09/ai-researchers-allege-machine-learning-alchemy/](https://reconsider.news/2018/05/09/ai-researchers-allege-machine-learning-alchemy/)]]

- Cost function is never convex!

- Hungry for data!

- Optimal architecture of DNN is still an open question!

- Many combinations of hyperparameters to be tuned or considered ...

---
## IV. Deep Neural Networks (DNN)<h0br> 
###  <img src="./img/neural.png" width="32px"/> Learning diagnosis and tips<h0br>
.pull-left[ <h0br>
- [Learning curve diagnosis](https://machinelearningmastery.com/learning-curves-for-diagnosing-machine-learning-model-performance/):

.center[ <hbr>
.caption[&nbsp; &nbsp; Too simple &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Need more training &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Overfitting] <br>
<img src="./img/diagnose1.png" width="110px" height ="92px"/> 
<img src="./img/diagnose2.png" width="110px" height ="92px"/> 
<img src="./img/diagnose3.png" width="110px" height ="92px"/> <br>

.caption[&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Easy validation &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Unrepresentative training data &nbsp; &nbsp; Unrepresentative validation data] <br>
<img src="./img/diagnose7.png" width="110px" height ="92px"/> 
<img src="./img/diagnose5.png" width="110px" height ="92px"/> 
<img src="./img/diagnose6.png" width="110px" height ="92px"/> <br>

.caption[Seem well fitted] <br>
<img src="./img/diagnose4.png" width="110px" height ="92px"/>] 
]
.pull-right[ <h0br>
- Start :
  - Weight initialization
  - Regularization
  - Dropout
  - Validation/minibatch size
  - Batch normalization ...
  
- During training: `$$\text{Start small}\\ \Downarrow\\ \text{Diagnosing}\\ \Updownarrow\\ \text{Adjusting}$$`

- Stop: diagnosing or early stopping ...

]

---
## IV. Deep Neural Networks (DNN)<h0br> 
###  <img src="./img/neural.png" width="32px"/> Some common types of ANN<h0br>
.pull-left[ <h0br>
- `$L=1$` (without hidden layer):
.left[<img src="./img/logRM.png" width="300px" height ="150px"/>]  <h0br>

- .stress[Convenlutional NN]: Images,  <br>

]

.pull-left[ <h0br>
- .stress[Recurrent NN (LSTM)]: TS or NLP,
.left[<img src="./img/LSTM.png" width="230px" height ="150px"/>]  <h0br>

- .stress[Transformers (attention)]: NLP,  <h0br>

.center[<img src="./img/attention.png" width="150px" height ="230px"/> <hbr> 
.caption[[Source: Attention is all you need.](https://arxiv.org/abs/1706.03762)]]
]

---

## IV. Deep Neural Networks (DNN)<h0br> 
###  <img src="./img/neural.png" width="32px"/> Comparison: Spam filtering <h0br>
.pull-left[ <h0br>
- .stress[Spam filtering]:

| Not_penalized| Penalized|       DNN|
|-------------:|---------:|---------:|
|     0.9336957| 0.9369565| 0.9413043|

]

- .stress[Mnist digit recognition]:

|    MLP|    DNN|
|------:|------:|
| 0.9272| 0.9785|

]

---
## Conclusion <h0br>

- .stress[Summary]:

- Linear Models: regression and classification.

- Some intuitions of gradient-based optimization: gradient descent.

- Regularization: constraint the magnitude of parameters, ovoid overfitting.

- DNN & comparisons.
  
--

- .stress[Some perspectives]:

- ML is not only optimization, quantity and quality of data matter!

- Interpretability is sometimes more important than predictability.
  
  - Universal method doesn't exist! It all comes down to trying.
  
  - Other (nonparametric) methods: trees, random forest, adaboost and xgboost...

---
count: false

&#128218; [I. Goodfellow, Y. Bengio, A. Courville. Deep Learning, MIT Press, 2016.](http://www.deeplearningbook.org)

&#128218; [T. Hastie, R. Tibshirani, J. Friedman. The Elements of Statistical Learning. Springer, 2009. ISBN:
978-0-387-84858-7.](https://link.springer.com/book/10.1007/978-0-387-84858-7)

&#128218; [A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin. Attention is all you need.  Advances in Neural Information Processing Systems 30 (NIPS 2017)](https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html)

&#128218; [Java T Point: https://www.javatpoint.com/applications-of-machine-learning](https://www.javatpoint.com/applications-of-machine-learning)

&#128218; [Java T Point: https://www.javatpoint.com/basic-concepts-in-machine-learning](https://www.javatpoint.com/basic-concepts-in-machine-learning)

&#128218; [D. Jurafsky and J. H. Martin. Speech and Language Processing. 2023](https://web.stanford.edu/~jurafsky/slp3/)

<h0br>

---
## Lab Class: Jupyter Notebook <h0br>

- Go to the following `github` for `notebook`:

.center[.large[<svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg> [https://github.com/hassothea/TeachingML](https://github.com/hassothea/TeachingML)]]