GradientCobra Python Library

Introduction to GradientCOBRA method

GradientCOBRA is a consensual aggregation method that combines a given number of regressors based on their predicted features (Has, 2023). The method is summarized in the following figure.

where

\({\bf r}(x)=(r_1(x),\dots, r_M(x))\) is the predicted features of observation \(x\).
\(K:\mathbb{R}_+\to\mathbb{R}_+\) is a regular kernel function that converts the “closeness” between predicted features of observation \(x_i\) and \(x\) into the associated weight for the aggregation. For any smoothing parameter \(h>0\), \(K_h(t)=K(t/h)\).

Constructing a consensual aggregation method is equivalent to estimating an appropriate smoothing parameter \(h\) that minimizes the so-called \(\kappa\)-fold cross validation error:

\[\phi(h)=\frac{1}{\kappa}\sum_{j=1}^{\kappa}\sum_{(x_i,y_i)\in F_j}\ell(\hat{y}_{-F_j}(x_i),y_i)\]

where

\(\ell\) is any regression loss function. By default, \(\ell(y_1,y_2)=(y_1-y_2)^2\).
\((F_j)_{j=1,2,\dots,\kappa}\) are the \(\kappa\) folds of the second of the training data used for aggregation.
\(\hat{y}_{-F_j}(x)\) is the prediction of \(x\) by aggregating predicted features on all folds \((F_k)_{k}\) except for \(F_j\).

GradientCOBRA method

class GradientCOBRA(random_state = None, learning_rate = 0.01, bandwidth_list = None, speed = ‘constant’, estimator_list = None, estimator_params = None, opt_method = “grad”, max_iter = int(300), opt_params = None, kernel = ‘radial’, kernel_exponent = 1.0, show_progress = True, loss_function = None, loss_weight = None, norm_constant = None):

Parameters

random_state: (default is None) for setting the random state of all the random generators in the class. This allows the method to produce the same result when the same random_state is used.
learning_rate: (default is 0.1) the learning rate in gradient descent algorithm for estimating the optimal bandwidth.
bandwidth_list: a list of bandwidth parameters for grid search algorithm (default = np.linspace(0.00001, 10, 300)).
speed: (default is constant) for adjusting the speed of the learning rate. It is helpful when the cost function is flat around the optimal value, changing the learning speed might help the algorithm to converge faster. It should be an element of [‘constant’, ‘linear’, ‘log’, ‘sqrt_root’, ‘quad’, ‘exp’].
estimator_list: (default is None) the list of intial estimators (machines as addressed in Biau et al. (2016)). If it is None, the intial learners including ‘linear_regression’, ‘ridge’, ‘lasso’, ‘tree’, and ‘random_forest’ are used with their default parameters. This argument should be a sublist of the following list: [‘linear_regression’, ‘knn’, ‘ridge’, ‘lasso’, ‘tree’, ‘random_forest’, ‘svm’, ‘sgd’, ‘bayesian_ridge’, ‘adaboost’, ‘gradient_boost’].
estimator_params: (default is None) is a dictionary containing the parameters of the basic estimators given in the estimator_list argument. It must be a dictionary with:
- key : the name of the basic estimator defined in estimator_list,
- value : a dictionary with (key, value) = (parameter, value).
opt_method: (default is “grad”) is an optimization algorithm for estimating the bandwidth parameter. It should be either “grid” (grid search) or “grad” (gradient descent for non-compactly supported kernels).
max_iter: (default = 100) maximum iteration of gradient descent algorithm.
opt_params: (default is None) is a dictionary of additional parameters for the optimization algorithm (both grid search and gradient descent). Its should contain some of the following keys:
- ‘epsilon’: stopping criterion for gradient descent algorithm (default = \(10^{-6}\)).
- ‘n_tries’: the number of tries for selecting initial position of gradient descent algorithm (default = 10)
- ‘start’: the initial value of the bandwidth parameter (default = None)
- ‘n_cv’: number of cross-validation folds (default = 10)
- ‘precision’ : the precision to estimate the gradient for gradient descent algorithm (default = \(10^{-10}\)).
kernel: (default is 'radial') the kernel function used for the aggregation. It should be an element of the list [‘exponential’, ‘gaussian’, ‘radial’, ‘cauchy’, ‘reverse_cosh’,‘epanechnikov’,‘biweight’, ‘triweight’, ‘triangular’, ‘cobra’, ‘naive’]. Some options such as 'gaussian' and 'radial' lead to the same radial kernel function. For 'cobra' or 'naive', they correspond to COBRA method by Biau et al. (2016).
kernel_exponent: (default is 1.0) is an exponential \(\alpha\) of the exponential and radial kernel function i.e., \(K(x) = \exp(|x|^{2\alpha})\). By default, alpha = 1.0.
show_progress: (default is True) a boolean defining whether or not to show the progress of the optimization algorithm for both grid search and gradient descent.
loss_function: (default is None) a function or string defining the cost function to be optimized for estimating the optimal bandwidth parameter. By defalut, the K-Fold cross-validation MSE is used. Otherwise, it must be either:
- a function of two argumetns (y_true, y_pred) or
- a string element of the list [‘mse’, ‘mae’, ‘mape’, ‘weighted_mse’]. If it is weighted_mse, one can define the weight for each training point using loss_weight argument below.
  - loss_weight: (default is None) a list of size \(n\) (number of training observations) defining the weight for each individual data point for the loss function. If it is None and the loss_function = weighted_mse, then a normalized weight \(W(i) = 1/\text{PDF}(i)\) is assigned to the individual \(i\) of the training data.
norm_constant: (default is None) a normalized constant used to scale the features in optimization algorithm. By defalut, they are

Returns:

self : returns an instance of self containing the optimal estimated bandwidth and other results of the algorithm.

Methods:

fit : fitting the aggregation method on the design features (original data or predicted features).
split_data : to split the data into \(D_k = \{(X_k,y_k)\}\) and \(D_{\ell} = \{(X_\ell,y_\ell)\}\) to train the basic estimators and perform aggregation respectively.
build_basic_estimators : build basic estimators for the aggregation. It is also possible to set the values of (hyper)parameters for each estimators.
load_predictions : to make predictions using the constructed basic estimators.
distances : to construct distance matrix according to the kernel function used for the aggregation.
kappa_cross_validation_error : the objective function to be minimized.
optimize_bandwidth : the optimization method used to estimate the optimal bendwidth parameter.
predict : for building prediction on the new observations using any given bendwidth or the estimated one.
draw_learning_curve : for plotting the graphic of learning algorithm (error vs parameter).

Installation of the library from `pypi`

gradientcobra can be installed from pypi using pip:

pip install gradientcobra

Implementation of the method

Importing some packages

# Metric of error
from sklearn.metrics import mean_squared_error, mean_absolute_error, mean_absolute_percentage_error

# Plotting figures
import matplotlib.pyplot as plt
from matplotlib import cm

# Import class GradientCOBRA from the gradientcobra library
from gradientcobra.gradientcobra import GradientCOBRA

import seaborn as sns
sns.set()

`GradientCOBRA` on simulated data

We simulate a regression data with \(1000\) observations and \(10\) inputs variables.

# For simulating dataset
from sklearn.datasets import make_regression

X1, y1 = make_regression(n_samples=1000, n_features=10, noise=2)

Now, let’s randomly split the simulated data into \(80\%-20\%\) training-testing data.

from sklearn.model_selection import train_test_split

X_train1, X_test1, y_train1, y_test1 = train_test_split(X1, y1, test_size=0.2)
print('shape: x_train = {} , x_train = {} , y_train = {} , y_test = {}'.format(
    X_train1.shape, 
    X_test1.shape, 
    y_train1.shape, 
    y_test1.shape))

shape: x_train = (800, 10) , x_train = (200, 10) , y_train = (800,) , y_test = (200,)

\(\bullet\) `GradientCOBRA` with default parameters

We create GradientCOBRA object called gc1 using GradientCOBRA class with the default parameters, then fit it to the training data.

gc1 = GradientCOBRA()
gc1_fit = gc1.fit(X_train1, y_train1)

* GD progress: iter: 1 / bw: 3.000 / grad: -26.306 / stop criter: 1.000 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 1 / bw: 3.100 / grad: -26.306 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 1 / bw: 3.100 / grad: -26.306 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 2 / bw: 3.100 / grad: -26.306 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 2 / bw: 3.100 / grad: -26.306 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 3 / bw: 3.100 / grad: -26.306 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 3 / bw: 3.100 / grad: -26.306 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 4 / bw: 3.100 / grad: -26.306 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 4 / bw: 3.100 / grad: -26.306 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 5 / bw: 3.100 / grad: 128.356 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 5 / bw: 3.100 / grad: 128.356 / stop criter: 26.306 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 6 / bw: 2.612 / grad: 28.396 / stop criter: 128.356 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 6 / bw: 2.612 / grad: 28.396 / stop criter: 128.356 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 7 / bw: 2.505 / grad: 10.042 / stop criter: 28.396 :   0%|          | 0/300 [00:00<?, ?it/s] * GD progress: iter: 7 / bw: 2.505 / grad: 10.042 / stop criter: 28.396 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 8 / bw: 2.467 / grad: 55.416 / stop criter: 10.042 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 8 / bw: 2.467 / grad: 55.416 / stop criter: 10.042 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 8 / bw: 2.467 / grad: 55.416 / stop criter: 10.042 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 9 / bw: 2.259 / grad: 49.916 / stop criter: 55.416 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 9 / bw: 2.259 / grad: 49.916 / stop criter: 55.416 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 10 / bw: 2.071 / grad: -90.447 / stop criter: 49.916 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 10 / bw: 2.071 / grad: -90.447 / stop criter: 49.916 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 11 / bw: 2.411 / grad: -129.252 / stop criter: 90.447 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 11 / bw: 2.411 / grad: -129.252 / stop criter: 90.447 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 12 / bw: 2.893 / grad: -53.371 / stop criter: 129.252 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 12 / bw: 2.893 / grad: -53.371 / stop criter: 129.252 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 13 / bw: 3.092 / grad: 19.150 / stop criter: 53.371 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]  * GD progress: iter: 13 / bw: 3.092 / grad: 19.150 / stop criter: 53.371 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 14 / bw: 3.020 / grad: -280.491 / stop criter: 19.150 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 14 / bw: 3.020 / grad: -280.491 / stop criter: 19.150 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 15 / bw: 4.055 / grad: 159.707 / stop criter: 280.491 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 15 / bw: 4.055 / grad: 159.707 / stop criter: 280.491 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 16 / bw: 3.472 / grad: -100.680 / stop criter: 159.707 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 16 / bw: 3.472 / grad: -100.680 / stop criter: 159.707 :   3%|▎         | 8/300 [00:00<00:04, 72.07it/s]* GD progress: iter: 16 / bw: 3.472 / grad: -100.680 / stop criter: 159.707 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 17 / bw: 3.836 / grad: 4.594 / stop criter: 100.680 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]   * GD progress: iter: 17 / bw: 3.836 / grad: 4.594 / stop criter: 100.680 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 18 / bw: 3.819 / grad: -69.075 / stop criter: 4.594 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 18 / bw: 3.819 / grad: -69.075 / stop criter: 4.594 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 19 / bw: 4.064 / grad: -8.764 / stop criter: 69.075 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 19 / bw: 4.064 / grad: -8.764 / stop criter: 69.075 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 20 / bw: 4.095 / grad: -92.812 / stop criter: 8.764 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 20 / bw: 4.095 / grad: -92.812 / stop criter: 8.764 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 21 / bw: 4.420 / grad: 157.962 / stop criter: 92.812 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 21 / bw: 4.420 / grad: 157.962 / stop criter: 92.812 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 22 / bw: 3.866 / grad: -46.266 / stop criter: 157.962 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 22 / bw: 3.866 / grad: -46.266 / stop criter: 157.962 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 23 / bw: 4.027 / grad: 15.219 / stop criter: 46.266 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]  * GD progress: iter: 23 / bw: 4.027 / grad: 15.219 / stop criter: 46.266 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 24 / bw: 3.975 / grad: -36.188 / stop criter: 15.219 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 24 / bw: 3.975 / grad: -36.188 / stop criter: 15.219 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 25 / bw: 4.098 / grad: -92.650 / stop criter: 36.188 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 25 / bw: 4.098 / grad: -92.650 / stop criter: 36.188 :   5%|▌         | 16/300 [00:00<00:03, 75.42it/s]* GD progress: iter: 25 / bw: 4.098 / grad: -92.650 / stop criter: 36.188 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 26 / bw: 4.410 / grad: 79.300 / stop criter: 92.650 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s] * GD progress: iter: 26 / bw: 4.410 / grad: 79.300 / stop criter: 92.650 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 27 / bw: 4.143 / grad: 190.957 / stop criter: 79.300 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 27 / bw: 4.143 / grad: 190.957 / stop criter: 79.300 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 28 / bw: 3.506 / grad: -34.380 / stop criter: 190.957 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 28 / bw: 3.506 / grad: -34.380 / stop criter: 190.957 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 29 / bw: 3.621 / grad: 126.892 / stop criter: 34.380 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s] * GD progress: iter: 29 / bw: 3.621 / grad: 126.892 / stop criter: 34.380 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 30 / bw: 3.201 / grad: 70.139 / stop criter: 126.892 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 30 / bw: 3.201 / grad: 70.139 / stop criter: 126.892 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 31 / bw: 2.972 / grad: 28.802 / stop criter: 70.139 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s] * GD progress: iter: 31 / bw: 2.972 / grad: 28.802 / stop criter: 70.139 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 32 / bw: 2.878 / grad: 81.526 / stop criter: 28.802 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 32 / bw: 2.878 / grad: 81.526 / stop criter: 28.802 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 33 / bw: 2.611 / grad: -51.967 / stop criter: 81.526 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 33 / bw: 2.611 / grad: -51.967 / stop criter: 81.526 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 34 / bw: 2.781 / grad: -140.370 / stop criter: 51.967 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 34 / bw: 2.781 / grad: -140.370 / stop criter: 51.967 :   8%|▊         | 25/300 [00:00<00:03, 80.46it/s]* GD progress: iter: 34 / bw: 2.781 / grad: -140.370 / stop criter: 51.967 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 35 / bw: 3.236 / grad: 100.179 / stop criter: 140.370 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 35 / bw: 3.236 / grad: 100.179 / stop criter: 140.370 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 36 / bw: 2.911 / grad: -57.622 / stop criter: 100.179 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 36 / bw: 2.911 / grad: -57.622 / stop criter: 100.179 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 37 / bw: 3.096 / grad: -125.374 / stop criter: 57.622 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 37 / bw: 3.096 / grad: -125.374 / stop criter: 57.622 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 38 / bw: 3.494 / grad: 52.032 / stop criter: 125.374 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s] * GD progress: iter: 38 / bw: 3.494 / grad: 52.032 / stop criter: 125.374 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 39 / bw: 3.329 / grad: -90.651 / stop criter: 52.032 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 39 / bw: 3.329 / grad: -90.651 / stop criter: 52.032 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 40 / bw: 3.613 / grad: -220.469 / stop criter: 90.651 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 40 / bw: 3.613 / grad: -220.469 / stop criter: 90.651 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 41 / bw: 4.299 / grad: -10.159 / stop criter: 220.469 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 41 / bw: 4.299 / grad: -10.159 / stop criter: 220.469 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 42 / bw: 4.331 / grad: 17.801 / stop criter: 10.159 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]  * GD progress: iter: 42 / bw: 4.331 / grad: 17.801 / stop criter: 10.159 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 43 / bw: 4.275 / grad: -90.448 / stop criter: 17.801 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 43 / bw: 4.275 / grad: -90.448 / stop criter: 17.801 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 44 / bw: 4.554 / grad: -168.250 / stop criter: 90.448 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 44 / bw: 4.554 / grad: -168.250 / stop criter: 90.448 :  11%|█▏        | 34/300 [00:00<00:03, 79.33it/s]* GD progress: iter: 44 / bw: 4.554 / grad: -168.250 / stop criter: 90.448 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 45 / bw: 5.066 / grad: 78.884 / stop criter: 168.250 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s] * GD progress: iter: 45 / bw: 5.066 / grad: 78.884 / stop criter: 168.250 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 46 / bw: 4.826 / grad: -165.897 / stop criter: 78.884 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 46 / bw: 4.826 / grad: -165.897 / stop criter: 78.884 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 47 / bw: 5.326 / grad: 158.954 / stop criter: 165.897 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 47 / bw: 5.326 / grad: 158.954 / stop criter: 165.897 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 48 / bw: 4.852 / grad: -143.880 / stop criter: 158.954 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 48 / bw: 4.852 / grad: -143.880 / stop criter: 158.954 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 49 / bw: 5.277 / grad: -40.236 / stop criter: 143.880 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s] * GD progress: iter: 49 / bw: 5.277 / grad: -40.236 / stop criter: 143.880 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 50 / bw: 5.395 / grad: -115.308 / stop criter: 40.236 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 50 / bw: 5.395 / grad: -115.308 / stop criter: 40.236 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 51 / bw: 5.732 / grad: 140.253 / stop criter: 115.308 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 51 / bw: 5.732 / grad: 140.253 / stop criter: 115.308 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 52 / bw: 5.322 / grad: -178.598 / stop criter: 140.253 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 52 / bw: 5.322 / grad: -178.598 / stop criter: 140.253 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 53 / bw: 5.839 / grad: -84.206 / stop criter: 178.598 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s] * GD progress: iter: 53 / bw: 5.839 / grad: -84.206 / stop criter: 178.598 :  15%|█▍        | 44/300 [00:00<00:02, 86.33it/s]* GD progress: iter: 53 / bw: 5.839 / grad: -84.206 / stop criter: 178.598 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 54 / bw: 6.081 / grad: -33.571 / stop criter: 84.206 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s] * GD progress: iter: 54 / bw: 6.081 / grad: -33.571 / stop criter: 84.206 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 55 / bw: 6.177 / grad: 33.531 / stop criter: 33.571 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s] * GD progress: iter: 55 / bw: 6.177 / grad: 33.531 / stop criter: 33.571 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 56 / bw: 6.081 / grad: 69.052 / stop criter: 33.531 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 56 / bw: 6.081 / grad: 69.052 / stop criter: 33.531 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 57 / bw: 5.885 / grad: -22.549 / stop criter: 69.052 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 57 / bw: 5.885 / grad: -22.549 / stop criter: 69.052 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 58 / bw: 5.949 / grad: 9.732 / stop criter: 22.549 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]  * GD progress: iter: 58 / bw: 5.949 / grad: 9.732 / stop criter: 22.549 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 59 / bw: 5.922 / grad: 123.023 / stop criter: 9.732 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 59 / bw: 5.922 / grad: 123.023 / stop criter: 9.732 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 60 / bw: 5.579 / grad: 77.229 / stop criter: 123.023 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 60 / bw: 5.579 / grad: 77.229 / stop criter: 123.023 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 61 / bw: 5.364 / grad: -32.236 / stop criter: 77.229 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 61 / bw: 5.364 / grad: -32.236 / stop criter: 77.229 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 62 / bw: 5.454 / grad: -98.447 / stop criter: 32.236 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 62 / bw: 5.454 / grad: -98.447 / stop criter: 32.236 :  18%|█▊        | 53/300 [00:00<00:02, 84.74it/s]* GD progress: iter: 62 / bw: 5.454 / grad: -98.447 / stop criter: 32.236 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 63 / bw: 5.725 / grad: 101.111 / stop criter: 98.447 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 63 / bw: 5.725 / grad: 101.111 / stop criter: 98.447 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 64 / bw: 5.447 / grad: 44.196 / stop criter: 101.111 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 64 / bw: 5.447 / grad: 44.196 / stop criter: 101.111 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 65 / bw: 5.326 / grad: -53.779 / stop criter: 44.196 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 65 / bw: 5.326 / grad: -53.779 / stop criter: 44.196 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 66 / bw: 5.473 / grad: -45.760 / stop criter: 53.779 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 66 / bw: 5.473 / grad: -45.760 / stop criter: 53.779 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 67 / bw: 5.596 / grad: -14.643 / stop criter: 45.760 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 67 / bw: 5.596 / grad: -14.643 / stop criter: 45.760 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 68 / bw: 5.636 / grad: 5.100 / stop criter: 14.643 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]  * GD progress: iter: 68 / bw: 5.636 / grad: 5.100 / stop criter: 14.643 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 69 / bw: 5.622 / grad: -36.486 / stop criter: 5.100 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 69 / bw: 5.622 / grad: -36.486 / stop criter: 5.100 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 70 / bw: 5.720 / grad: 109.553 / stop criter: 36.486 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 70 / bw: 5.720 / grad: 109.553 / stop criter: 36.486 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 71 / bw: 5.430 / grad: -1.138 / stop criter: 109.553 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 71 / bw: 5.430 / grad: -1.138 / stop criter: 109.553 :  21%|██        | 62/300 [00:00<00:02, 84.19it/s]* GD progress: iter: 71 / bw: 5.430 / grad: -1.138 / stop criter: 109.553 :  24%|██▎       | 71/300 [00:00<00:03, 74.70it/s]* GD progress: iter: 72 / bw: 5.433 / grad: 34.203 / stop criter: 1.138 :  24%|██▎       | 71/300 [00:00<00:03, 74.70it/s]  * GD progress: iter: 72 / bw: 5.433 / grad: 34.203 / stop criter: 1.138 :  24%|██▎       | 71/300 [00:00<00:03, 74.70it/s]* GD progress: iter: 73 / bw: 5.344 / grad: -209.600 / stop criter: 34.203 :  24%|██▎       | 71/300 [00:00<00:03, 74.70it/s]* GD progress: iter: 73 / bw: 5.344 / grad: -209.600 / stop criter: 34.203 :  24%|██▎       | 71/300 [00:00<00:03, 74.70it/s]* GD progress: iter: 74 / bw: 5.882 / grad: 65.372 / stop criter: 209.600 :  24%|██▎       | 71/300 [00:00<00:03, 74.70it/s] * GD progress: iter: 74 / bw: 5.882 / grad: 65.372 / stop criter: 209.600 :  24%|██▎       | 71/300 [00:00<00:03, 74.70it/s]* GD progress: iter: 75 / bw: 5.716 / grad: -137.753 / stop criter: 65.372 :  24%|██▎       | 71/300 [00:00<00:03, 74.70it/s]* GD progress: iter: 75 / bw: 5.716 / grad: -137.753 / stop criter: 65.372 :  24%|██▎       | 71/300 [00:00<00:03, 74.70it/s]* GD progress: iter: 76 / bw: 6.063 / grad: 178.385 / stop criter: 137.753 :  24%|██▎       | 71/300 [00:01<00:03, 74.70it/s]* GD progress: iter: 76 / bw: 6.063 / grad: 178.385 / stop criter: 137.753 :  24%|██▎       | 71/300 [00:01<00:03, 74.70it/s]* GD progress: iter: 77 / bw: 5.618 / grad: -88.311 / stop criter: 178.385 :  24%|██▎       | 71/300 [00:01<00:03, 74.70it/s]* GD progress: iter: 77 / bw: 5.618 / grad: -88.311 / stop criter: 178.385 :  24%|██▎       | 71/300 [00:01<00:03, 74.70it/s]* GD progress: iter: 78 / bw: 5.836 / grad: -154.952 / stop criter: 88.311 :  24%|██▎       | 71/300 [00:01<00:03, 74.70it/s]* GD progress: iter: 78 / bw: 5.836 / grad: -154.952 / stop criter: 88.311 :  24%|██▎       | 71/300 [00:01<00:03, 74.70it/s]* GD progress: iter: 79 / bw: 6.215 / grad: -7.427 / stop criter: 154.952 :  24%|██▎       | 71/300 [00:01<00:03, 74.70it/s] * GD progress: iter: 79 / bw: 6.215 / grad: -7.427 / stop criter: 154.952 :  24%|██▎       | 71/300 [00:01<00:03, 74.70it/s]* GD progress: iter: 79 / bw: 6.215 / grad: -7.427 / stop criter: 154.952 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 80 / bw: 6.233 / grad: 75.369 / stop criter: 7.427 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]  * GD progress: iter: 80 / bw: 6.233 / grad: 75.369 / stop criter: 7.427 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 81 / bw: 6.049 / grad: -45.054 / stop criter: 75.369 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 81 / bw: 6.049 / grad: -45.054 / stop criter: 75.369 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 82 / bw: 6.158 / grad: 33.833 / stop criter: 45.054 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s] * GD progress: iter: 82 / bw: 6.158 / grad: 33.833 / stop criter: 45.054 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 83 / bw: 6.077 / grad: -21.859 / stop criter: 33.833 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 83 / bw: 6.077 / grad: -21.859 / stop criter: 33.833 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 84 / bw: 6.129 / grad: 89.837 / stop criter: 21.859 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s] * GD progress: iter: 84 / bw: 6.129 / grad: 89.837 / stop criter: 21.859 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 85 / bw: 5.918 / grad: -99.796 / stop criter: 89.837 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 85 / bw: 5.918 / grad: -99.796 / stop criter: 89.837 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 86 / bw: 6.150 / grad: 187.782 / stop criter: 99.796 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 86 / bw: 6.150 / grad: 187.782 / stop criter: 99.796 :  26%|██▋       | 79/300 [00:01<00:03, 62.27it/s]* GD progress: iter: 86 / bw: 6.150 / grad: 187.782 / stop criter: 99.796 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 87 / bw: 5.718 / grad: 59.068 / stop criter: 187.782 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 87 / bw: 5.718 / grad: 59.068 / stop criter: 187.782 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 88 / bw: 5.583 / grad: 16.371 / stop criter: 59.068 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s] * GD progress: iter: 88 / bw: 5.583 / grad: 16.371 / stop criter: 59.068 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 89 / bw: 5.546 / grad: -252.343 / stop criter: 16.371 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 89 / bw: 5.546 / grad: -252.343 / stop criter: 16.371 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 90 / bw: 6.120 / grad: 145.822 / stop criter: 252.343 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 90 / bw: 6.120 / grad: 145.822 / stop criter: 252.343 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 91 / bw: 5.792 / grad: 30.679 / stop criter: 145.822 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s] * GD progress: iter: 91 / bw: 5.792 / grad: 30.679 / stop criter: 145.822 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 92 / bw: 5.723 / grad: 268.080 / stop criter: 30.679 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 92 / bw: 5.723 / grad: 268.080 / stop criter: 30.679 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 93 / bw: 5.125 / grad: 157.522 / stop criter: 268.080 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 93 / bw: 5.125 / grad: 157.522 / stop criter: 268.080 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 94 / bw: 4.774 / grad: -220.549 / stop criter: 157.522 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 94 / bw: 4.774 / grad: -220.549 / stop criter: 157.522 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 95 / bw: 5.266 / grad: -102.911 / stop criter: 220.549 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 95 / bw: 5.266 / grad: -102.911 / stop criter: 220.549 :  29%|██▊       | 86/300 [00:01<00:03, 61.62it/s]* GD progress: iter: 95 / bw: 5.266 / grad: -102.911 / stop criter: 220.549 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 96 / bw: 5.493 / grad: 33.570 / stop criter: 102.911 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]  * GD progress: iter: 96 / bw: 5.493 / grad: 33.570 / stop criter: 102.911 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 97 / bw: 5.419 / grad: 87.952 / stop criter: 33.570 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s] * GD progress: iter: 97 / bw: 5.419 / grad: 87.952 / stop criter: 33.570 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 98 / bw: 5.227 / grad: 266.181 / stop criter: 87.952 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 98 / bw: 5.227 / grad: 266.181 / stop criter: 87.952 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 99 / bw: 4.644 / grad: -99.543 / stop criter: 266.181 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 99 / bw: 4.644 / grad: -99.543 / stop criter: 266.181 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 100 / bw: 4.862 / grad: -17.591 / stop criter: 99.543 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 100 / bw: 4.862 / grad: -17.591 / stop criter: 99.543 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 101 / bw: 4.900 / grad: -214.269 / stop criter: 17.591 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 101 / bw: 4.900 / grad: -214.269 / stop criter: 17.591 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 102 / bw: 5.364 / grad: 165.417 / stop criter: 214.269 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 102 / bw: 5.364 / grad: 165.417 / stop criter: 214.269 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 103 / bw: 5.006 / grad: 122.717 / stop criter: 165.417 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 103 / bw: 5.006 / grad: 122.717 / stop criter: 165.417 :  32%|███▏      | 95/300 [00:01<00:03, 67.44it/s]* GD progress: iter: 103 / bw: 5.006 / grad: 122.717 / stop criter: 165.417 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 104 / bw: 4.743 / grad: -96.578 / stop criter: 122.717 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 104 / bw: 4.743 / grad: -96.578 / stop criter: 122.717 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 105 / bw: 4.950 / grad: -120.068 / stop criter: 96.578 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 105 / bw: 4.950 / grad: -120.068 / stop criter: 96.578 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 106 / bw: 5.205 / grad: -62.619 / stop criter: 120.068 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 106 / bw: 5.205 / grad: -62.619 / stop criter: 120.068 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 107 / bw: 5.338 / grad: -239.014 / stop criter: 62.619 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 107 / bw: 5.338 / grad: -239.014 / stop criter: 62.619 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 108 / bw: 5.845 / grad: 15.942 / stop criter: 239.014 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s] * GD progress: iter: 108 / bw: 5.845 / grad: 15.942 / stop criter: 239.014 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 109 / bw: 5.811 / grad: -237.221 / stop criter: 15.942 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 109 / bw: 5.811 / grad: -237.221 / stop criter: 15.942 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 110 / bw: 6.309 / grad: -217.566 / stop criter: 237.221 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 110 / bw: 6.309 / grad: -217.566 / stop criter: 237.221 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 111 / bw: 6.762 / grad: 77.081 / stop criter: 217.566 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]  * GD progress: iter: 111 / bw: 6.762 / grad: 77.081 / stop criter: 217.566 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 112 / bw: 6.602 / grad: -39.334 / stop criter: 77.081 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 112 / bw: 6.602 / grad: -39.334 / stop criter: 77.081 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 113 / bw: 6.683 / grad: 130.074 / stop criter: 39.334 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 113 / bw: 6.683 / grad: 130.074 / stop criter: 39.334 :  34%|███▍      | 103/300 [00:01<00:02, 66.25it/s]* GD progress: iter: 113 / bw: 6.683 / grad: 130.074 / stop criter: 39.334 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 114 / bw: 6.417 / grad: -103.932 / stop criter: 130.074 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 114 / bw: 6.417 / grad: -103.932 / stop criter: 130.074 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 115 / bw: 6.627 / grad: -221.628 / stop criter: 103.932 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 115 / bw: 6.627 / grad: -221.628 / stop criter: 103.932 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 116 / bw: 7.070 / grad: 143.353 / stop criter: 221.628 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s] * GD progress: iter: 116 / bw: 7.070 / grad: 143.353 / stop criter: 221.628 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 117 / bw: 6.784 / grad: -142.053 / stop criter: 143.353 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 117 / bw: 6.784 / grad: -142.053 / stop criter: 143.353 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 118 / bw: 7.065 / grad: 127.399 / stop criter: 142.053 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s] * GD progress: iter: 118 / bw: 7.065 / grad: 127.399 / stop criter: 142.053 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 119 / bw: 6.815 / grad: 70.998 / stop criter: 127.399 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s] * GD progress: iter: 119 / bw: 6.815 / grad: 70.998 / stop criter: 127.399 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 120 / bw: 6.677 / grad: 152.601 / stop criter: 70.998 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 120 / bw: 6.677 / grad: 152.601 / stop criter: 70.998 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 121 / bw: 6.382 / grad: -342.450 / stop criter: 152.601 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 121 / bw: 6.382 / grad: -342.450 / stop criter: 152.601 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 122 / bw: 7.046 / grad: -90.184 / stop criter: 342.450 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s] * GD progress: iter: 122 / bw: 7.046 / grad: -90.184 / stop criter: 342.450 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 123 / bw: 7.219 / grad: -238.491 / stop criter: 90.184 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 123 / bw: 7.219 / grad: -238.491 / stop criter: 90.184 :  38%|███▊      | 113/300 [00:01<00:02, 73.36it/s]* GD progress: iter: 123 / bw: 7.219 / grad: -238.491 / stop criter: 90.184 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 124 / bw: 7.676 / grad: 207.090 / stop criter: 238.491 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 124 / bw: 7.676 / grad: 207.090 / stop criter: 238.491 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 125 / bw: 7.279 / grad: 78.937 / stop criter: 207.090 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s] * GD progress: iter: 125 / bw: 7.279 / grad: 78.937 / stop criter: 207.090 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 126 / bw: 7.129 / grad: 131.022 / stop criter: 78.937 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 126 / bw: 7.129 / grad: 131.022 / stop criter: 78.937 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 127 / bw: 6.880 / grad: 43.907 / stop criter: 131.022 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 127 / bw: 6.880 / grad: 43.907 / stop criter: 131.022 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 128 / bw: 6.796 / grad: -29.351 / stop criter: 43.907 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 128 / bw: 6.796 / grad: -29.351 / stop criter: 43.907 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 129 / bw: 6.852 / grad: -12.140 / stop criter: 29.351 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 129 / bw: 6.852 / grad: -12.140 / stop criter: 29.351 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 130 / bw: 6.875 / grad: 30.047 / stop criter: 12.140 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s] * GD progress: iter: 130 / bw: 6.875 / grad: 30.047 / stop criter: 12.140 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 131 / bw: 6.819 / grad: 32.725 / stop criter: 30.047 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 131 / bw: 6.819 / grad: 32.725 / stop criter: 30.047 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 132 / bw: 6.758 / grad: -98.945 / stop criter: 32.725 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 132 / bw: 6.758 / grad: -98.945 / stop criter: 32.725 :  41%|████      | 123/300 [00:01<00:02, 79.68it/s]* GD progress: iter: 132 / bw: 6.758 / grad: -98.945 / stop criter: 32.725 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 133 / bw: 6.942 / grad: -83.748 / stop criter: 98.945 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 133 / bw: 6.942 / grad: -83.748 / stop criter: 98.945 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 134 / bw: 7.096 / grad: 44.525 / stop criter: 83.748 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s] * GD progress: iter: 134 / bw: 7.096 / grad: 44.525 / stop criter: 83.748 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 135 / bw: 7.014 / grad: -133.431 / stop criter: 44.525 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 135 / bw: 7.014 / grad: -133.431 / stop criter: 44.525 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 136 / bw: 7.258 / grad: -19.101 / stop criter: 133.431 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 136 / bw: 7.258 / grad: -19.101 / stop criter: 133.431 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 137 / bw: 7.292 / grad: 11.152 / stop criter: 19.101 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]  * GD progress: iter: 137 / bw: 7.292 / grad: 11.152 / stop criter: 19.101 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 138 / bw: 7.272 / grad: 24.977 / stop criter: 11.152 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 138 / bw: 7.272 / grad: 24.977 / stop criter: 11.152 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 139 / bw: 7.227 / grad: 65.267 / stop criter: 24.977 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 139 / bw: 7.227 / grad: 65.267 / stop criter: 24.977 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 140 / bw: 7.111 / grad: 146.758 / stop criter: 65.267 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 140 / bw: 7.111 / grad: 146.758 / stop criter: 65.267 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 141 / bw: 6.848 / grad: -105.617 / stop criter: 146.758 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 141 / bw: 6.848 / grad: -105.617 / stop criter: 146.758 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 142 / bw: 7.037 / grad: -35.572 / stop criter: 105.617 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s] * GD progress: iter: 142 / bw: 7.037 / grad: -35.572 / stop criter: 105.617 :  44%|████▍     | 132/300 [00:01<00:02, 81.50it/s]* GD progress: iter: 142 / bw: 7.037 / grad: -35.572 / stop criter: 105.617 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 143 / bw: 7.100 / grad: 78.114 / stop criter: 35.572 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]  * GD progress: iter: 143 / bw: 7.100 / grad: 78.114 / stop criter: 35.572 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 144 / bw: 6.962 / grad: 151.373 / stop criter: 78.114 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 144 / bw: 6.962 / grad: 151.373 / stop criter: 78.114 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 145 / bw: 6.696 / grad: 39.127 / stop criter: 151.373 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 145 / bw: 6.696 / grad: 39.127 / stop criter: 151.373 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 146 / bw: 6.628 / grad: -17.509 / stop criter: 39.127 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 146 / bw: 6.628 / grad: -17.509 / stop criter: 39.127 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 147 / bw: 6.658 / grad: 62.070 / stop criter: 17.509 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s] * GD progress: iter: 147 / bw: 6.658 / grad: 62.070 / stop criter: 17.509 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 148 / bw: 6.551 / grad: 67.325 / stop criter: 62.070 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 148 / bw: 6.551 / grad: 67.325 / stop criter: 62.070 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 149 / bw: 6.435 / grad: 116.948 / stop criter: 67.325 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 149 / bw: 6.435 / grad: 116.948 / stop criter: 67.325 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 150 / bw: 6.234 / grad: 159.892 / stop criter: 116.948 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 150 / bw: 6.234 / grad: 159.892 / stop criter: 116.948 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 151 / bw: 5.959 / grad: 24.525 / stop criter: 159.892 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s] * GD progress: iter: 151 / bw: 5.959 / grad: 24.525 / stop criter: 159.892 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 152 / bw: 5.917 / grad: 18.531 / stop criter: 24.525 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s] * GD progress: iter: 152 / bw: 5.917 / grad: 18.531 / stop criter: 24.525 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 153 / bw: 5.885 / grad: 118.354 / stop criter: 18.531 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 153 / bw: 5.885 / grad: 118.354 / stop criter: 18.531 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 154 / bw: 5.682 / grad: 105.508 / stop criter: 118.354 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 154 / bw: 5.682 / grad: 105.508 / stop criter: 118.354 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 155 / bw: 5.501 / grad: 108.074 / stop criter: 105.508 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 155 / bw: 5.501 / grad: 108.074 / stop criter: 105.508 :  47%|████▋     | 142/300 [00:01<00:01, 84.10it/s]* GD progress: iter: 155 / bw: 5.501 / grad: 108.074 / stop criter: 105.508 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s]* GD progress: iter: 156 / bw: 5.315 / grad: 43.242 / stop criter: 108.074 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s] * GD progress: iter: 156 / bw: 5.315 / grad: 43.242 / stop criter: 108.074 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s]* GD progress: iter: 157 / bw: 5.241 / grad: -225.629 / stop criter: 43.242 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s]* GD progress: iter: 157 / bw: 5.241 / grad: -225.629 / stop criter: 43.242 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s]* GD progress: iter: 158 / bw: 5.628 / grad: 110.111 / stop criter: 225.629 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s]* GD progress: iter: 158 / bw: 5.628 / grad: 110.111 / stop criter: 225.629 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s]* GD progress: iter: 159 / bw: 5.441 / grad: -157.832 / stop criter: 110.111 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s]* GD progress: iter: 159 / bw: 5.441 / grad: -157.832 / stop criter: 110.111 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s]* GD progress: iter: 160 / bw: 5.707 / grad: 149.315 / stop criter: 157.832 :  52%|█████▏    | 155/300 [00:01<00:01, 95.06it/s] * GD progress: iter: 160 / bw: 5.707 / grad: 149.315 / stop criter: 157.832 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 161 / bw: 5.458 / grad: 156.112 / stop criter: 149.315 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 161 / bw: 5.458 / grad: 156.112 / stop criter: 149.315 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 162 / bw: 5.200 / grad: -7.752 / stop criter: 156.112 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s] * GD progress: iter: 162 / bw: 5.200 / grad: -7.752 / stop criter: 156.112 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 163 / bw: 5.213 / grad: -78.201 / stop criter: 7.752 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s] * GD progress: iter: 163 / bw: 5.213 / grad: -78.201 / stop criter: 7.752 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 164 / bw: 5.341 / grad: -79.861 / stop criter: 78.201 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 164 / bw: 5.341 / grad: -79.861 / stop criter: 78.201 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 165 / bw: 5.471 / grad: 1.867 / stop criter: 79.861 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]  * GD progress: iter: 165 / bw: 5.471 / grad: 1.867 / stop criter: 79.861 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 166 / bw: 5.468 / grad: -2.683 / stop criter: 1.867 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 166 / bw: 5.468 / grad: -2.683 / stop criter: 1.867 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 167 / bw: 5.473 / grad: -194.413 / stop criter: 2.683 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 167 / bw: 5.473 / grad: -194.413 / stop criter: 2.683 :  52%|█████▏    | 155/300 [00:02<00:01, 95.06it/s]* GD progress: iter: 167 / bw: 5.473 / grad: -194.413 / stop criter: 2.683 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 168 / bw: 5.784 / grad: -65.613 / stop criter: 194.413 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 168 / bw: 5.784 / grad: -65.613 / stop criter: 194.413 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 169 / bw: 5.889 / grad: -130.916 / stop criter: 65.613 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 169 / bw: 5.889 / grad: -130.916 / stop criter: 65.613 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 170 / bw: 6.099 / grad: 274.525 / stop criter: 130.916 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 170 / bw: 6.099 / grad: 274.525 / stop criter: 130.916 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 171 / bw: 5.659 / grad: 52.343 / stop criter: 274.525 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s] * GD progress: iter: 171 / bw: 5.659 / grad: 52.343 / stop criter: 274.525 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 172 / bw: 5.576 / grad: -62.908 / stop criter: 52.343 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 172 / bw: 5.576 / grad: -62.908 / stop criter: 52.343 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 173 / bw: 5.676 / grad: 107.296 / stop criter: 62.908 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 173 / bw: 5.676 / grad: 107.296 / stop criter: 62.908 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 174 / bw: 5.507 / grad: 0.721 / stop criter: 107.296 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s] * GD progress: iter: 174 / bw: 5.507 / grad: 0.721 / stop criter: 107.296 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 175 / bw: 5.506 / grad: -175.566 / stop criter: 0.721 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 175 / bw: 5.506 / grad: -175.566 / stop criter: 0.721 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 176 / bw: 5.779 / grad: 124.085 / stop criter: 175.566 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 176 / bw: 5.779 / grad: 124.085 / stop criter: 175.566 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 177 / bw: 5.588 / grad: -81.974 / stop criter: 124.085 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 177 / bw: 5.588 / grad: -81.974 / stop criter: 124.085 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 178 / bw: 5.713 / grad: 172.011 / stop criter: 81.974 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s] * GD progress: iter: 178 / bw: 5.713 / grad: 172.011 / stop criter: 81.974 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 179 / bw: 5.454 / grad: 13.807 / stop criter: 172.011 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 179 / bw: 5.454 / grad: 13.807 / stop criter: 172.011 :  56%|█████▌    | 167/300 [00:02<00:01, 101.96it/s]* GD progress: iter: 179 / bw: 5.454 / grad: 13.807 / stop criter: 172.011 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 180 / bw: 5.433 / grad: -0.940 / stop criter: 13.807 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s] * GD progress: iter: 180 / bw: 5.433 / grad: -0.940 / stop criter: 13.807 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 181 / bw: 5.434 / grad: 15.126 / stop criter: 0.940 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s] * GD progress: iter: 181 / bw: 5.434 / grad: 15.126 / stop criter: 0.940 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 182 / bw: 5.412 / grad: -129.078 / stop criter: 15.126 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 182 / bw: 5.412 / grad: -129.078 / stop criter: 15.126 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 183 / bw: 5.601 / grad: -139.825 / stop criter: 129.078 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 183 / bw: 5.601 / grad: -139.825 / stop criter: 129.078 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 184 / bw: 5.803 / grad: -132.702 / stop criter: 139.825 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 184 / bw: 5.803 / grad: -132.702 / stop criter: 139.825 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 185 / bw: 5.996 / grad: 129.594 / stop criter: 132.702 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s] * GD progress: iter: 185 / bw: 5.996 / grad: 129.594 / stop criter: 132.702 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 186 / bw: 5.808 / grad: 30.519 / stop criter: 129.594 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s] * GD progress: iter: 186 / bw: 5.808 / grad: 30.519 / stop criter: 129.594 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 187 / bw: 5.764 / grad: -2.166 / stop criter: 30.519 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s] * GD progress: iter: 187 / bw: 5.764 / grad: -2.166 / stop criter: 30.519 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 188 / bw: 5.767 / grad: 206.132 / stop criter: 2.166 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 188 / bw: 5.767 / grad: 206.132 / stop criter: 2.166 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 189 / bw: 5.475 / grad: 82.967 / stop criter: 206.132 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 189 / bw: 5.475 / grad: 82.967 / stop criter: 206.132 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 190 / bw: 5.358 / grad: -68.424 / stop criter: 82.967 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 190 / bw: 5.358 / grad: -68.424 / stop criter: 82.967 :  60%|█████▉    | 179/300 [00:02<00:01, 102.52it/s]* GD progress: iter: 190 / bw: 5.358 / grad: -68.424 / stop criter: 82.967 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 191 / bw: 5.454 / grad: -127.950 / stop criter: 68.424 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 191 / bw: 5.454 / grad: -127.950 / stop criter: 68.424 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 192 / bw: 5.632 / grad: 0.844 / stop criter: 127.950 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]  * GD progress: iter: 192 / bw: 5.632 / grad: 0.844 / stop criter: 127.950 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 193 / bw: 5.631 / grad: -173.025 / stop criter: 0.844 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 193 / bw: 5.631 / grad: -173.025 / stop criter: 0.844 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 194 / bw: 5.869 / grad: 179.265 / stop criter: 173.025 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 194 / bw: 5.869 / grad: 179.265 / stop criter: 173.025 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 195 / bw: 5.625 / grad: -82.556 / stop criter: 179.265 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 195 / bw: 5.625 / grad: -82.556 / stop criter: 179.265 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 196 / bw: 5.736 / grad: 144.252 / stop criter: 82.556 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s] * GD progress: iter: 196 / bw: 5.736 / grad: 144.252 / stop criter: 82.556 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 197 / bw: 5.544 / grad: -14.350 / stop criter: 144.252 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 197 / bw: 5.544 / grad: -14.350 / stop criter: 144.252 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 198 / bw: 5.563 / grad: -103.269 / stop criter: 14.350 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 198 / bw: 5.563 / grad: -103.269 / stop criter: 14.350 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 199 / bw: 5.698 / grad: 40.812 / stop criter: 103.269 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s] * GD progress: iter: 199 / bw: 5.698 / grad: 40.812 / stop criter: 103.269 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 200 / bw: 5.644 / grad: -129.579 / stop criter: 40.812 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 200 / bw: 5.644 / grad: -129.579 / stop criter: 40.812 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 201 / bw: 5.812 / grad: -57.843 / stop criter: 129.579 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 201 / bw: 5.812 / grad: -57.843 / stop criter: 129.579 :  63%|██████▎   | 190/300 [00:02<00:01, 104.48it/s]* GD progress: iter: 201 / bw: 5.812 / grad: -57.843 / stop criter: 129.579 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s] * GD progress: iter: 202 / bw: 5.887 / grad: 56.723 / stop criter: 57.843 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]  * GD progress: iter: 202 / bw: 5.887 / grad: 56.723 / stop criter: 57.843 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 203 / bw: 5.814 / grad: 101.647 / stop criter: 56.723 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 203 / bw: 5.814 / grad: 101.647 / stop criter: 56.723 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 204 / bw: 5.685 / grad: 177.864 / stop criter: 101.647 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 204 / bw: 5.685 / grad: 177.864 / stop criter: 101.647 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 205 / bw: 5.459 / grad: -113.342 / stop criter: 177.864 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 205 / bw: 5.459 / grad: -113.342 / stop criter: 177.864 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 206 / bw: 5.603 / grad: -155.040 / stop criter: 113.342 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 206 / bw: 5.603 / grad: -155.040 / stop criter: 113.342 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 207 / bw: 5.798 / grad: -215.229 / stop criter: 155.040 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 207 / bw: 5.798 / grad: -215.229 / stop criter: 155.040 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 208 / bw: 6.069 / grad: 7.625 / stop criter: 215.229 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]   * GD progress: iter: 208 / bw: 6.069 / grad: 7.625 / stop criter: 215.229 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 209 / bw: 6.059 / grad: 115.324 / stop criter: 7.625 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 209 / bw: 6.059 / grad: 115.324 / stop criter: 7.625 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 210 / bw: 5.915 / grad: -15.096 / stop criter: 115.324 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 210 / bw: 5.915 / grad: -15.096 / stop criter: 115.324 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 211 / bw: 5.934 / grad: -82.492 / stop criter: 15.096 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s] * GD progress: iter: 211 / bw: 5.934 / grad: -82.492 / stop criter: 15.096 :  67%|██████▋   | 201/300 [00:02<00:01, 96.72it/s]* GD progress: iter: 211 / bw: 5.934 / grad: -82.492 / stop criter: 15.096 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 212 / bw: 6.036 / grad: -270.990 / stop criter: 82.492 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 212 / bw: 6.036 / grad: -270.990 / stop criter: 82.492 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 213 / bw: 6.370 / grad: 100.863 / stop criter: 270.990 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 213 / bw: 6.370 / grad: 100.863 / stop criter: 270.990 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 214 / bw: 6.246 / grad: -320.825 / stop criter: 100.863 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 214 / bw: 6.246 / grad: -320.825 / stop criter: 100.863 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 215 / bw: 6.637 / grad: -3.941 / stop criter: 320.825 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]  * GD progress: iter: 215 / bw: 6.637 / grad: -3.941 / stop criter: 320.825 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 216 / bw: 6.642 / grad: 215.161 / stop criter: 3.941 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s] * GD progress: iter: 216 / bw: 6.642 / grad: 215.161 / stop criter: 3.941 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 217 / bw: 6.382 / grad: 140.407 / stop criter: 215.161 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 217 / bw: 6.382 / grad: 140.407 / stop criter: 215.161 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 218 / bw: 6.214 / grad: 72.837 / stop criter: 140.407 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s] * GD progress: iter: 218 / bw: 6.214 / grad: 72.837 / stop criter: 140.407 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 219 / bw: 6.127 / grad: -73.000 / stop criter: 72.837 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 219 / bw: 6.127 / grad: -73.000 / stop criter: 72.837 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 220 / bw: 6.214 / grad: -49.542 / stop criter: 73.000 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 220 / bw: 6.214 / grad: -49.542 / stop criter: 73.000 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 221 / bw: 6.273 / grad: 19.428 / stop criter: 49.542 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s] * GD progress: iter: 221 / bw: 6.273 / grad: 19.428 / stop criter: 49.542 :  70%|███████   | 211/300 [00:02<00:00, 91.19it/s]* GD progress: iter: 221 / bw: 6.273 / grad: 19.428 / stop criter: 49.542 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 222 / bw: 6.250 / grad: -78.365 / stop criter: 19.428 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 222 / bw: 6.250 / grad: -78.365 / stop criter: 19.428 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 223 / bw: 6.342 / grad: -77.334 / stop criter: 78.365 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 223 / bw: 6.342 / grad: -77.334 / stop criter: 78.365 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 224 / bw: 6.432 / grad: 30.656 / stop criter: 77.334 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s] * GD progress: iter: 224 / bw: 6.432 / grad: 30.656 / stop criter: 77.334 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 225 / bw: 6.396 / grad: 59.344 / stop criter: 30.656 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 225 / bw: 6.396 / grad: 59.344 / stop criter: 30.656 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 226 / bw: 6.328 / grad: -87.608 / stop criter: 59.344 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 226 / bw: 6.328 / grad: -87.608 / stop criter: 59.344 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 227 / bw: 6.429 / grad: 45.542 / stop criter: 87.608 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s] * GD progress: iter: 227 / bw: 6.429 / grad: 45.542 / stop criter: 87.608 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 228 / bw: 6.377 / grad: 64.305 / stop criter: 45.542 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 228 / bw: 6.377 / grad: 64.305 / stop criter: 45.542 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 229 / bw: 6.304 / grad: -6.446 / stop criter: 64.305 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 229 / bw: 6.304 / grad: -6.446 / stop criter: 64.305 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 230 / bw: 6.312 / grad: 296.656 / stop criter: 6.446 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 230 / bw: 6.312 / grad: 296.656 / stop criter: 6.446 :  74%|███████▎  | 221/300 [00:02<00:00, 85.45it/s]* GD progress: iter: 230 / bw: 6.312 / grad: 296.656 / stop criter: 6.446 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 231 / bw: 5.981 / grad: 62.628 / stop criter: 296.656 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 231 / bw: 5.981 / grad: 62.628 / stop criter: 296.656 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 232 / bw: 5.912 / grad: -105.383 / stop criter: 62.628 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 232 / bw: 5.912 / grad: -105.383 / stop criter: 62.628 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 233 / bw: 6.028 / grad: -242.399 / stop criter: 105.383 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 233 / bw: 6.028 / grad: -242.399 / stop criter: 105.383 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 234 / bw: 6.293 / grad: 2.763 / stop criter: 242.399 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]   * GD progress: iter: 234 / bw: 6.293 / grad: 2.763 / stop criter: 242.399 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 235 / bw: 6.290 / grad: 76.228 / stop criter: 2.763 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s] * GD progress: iter: 235 / bw: 6.290 / grad: 76.228 / stop criter: 2.763 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 236 / bw: 6.207 / grad: 108.131 / stop criter: 76.228 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 236 / bw: 6.207 / grad: 108.131 / stop criter: 76.228 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 237 / bw: 6.090 / grad: 51.036 / stop criter: 108.131 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 237 / bw: 6.090 / grad: 51.036 / stop criter: 108.131 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 238 / bw: 6.035 / grad: 6.856 / stop criter: 51.036 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]  * GD progress: iter: 238 / bw: 6.035 / grad: 6.856 / stop criter: 51.036 :  77%|███████▋  | 230/300 [00:02<00:00, 75.85it/s]* GD progress: iter: 238 / bw: 6.035 / grad: 6.856 / stop criter: 51.036 :  79%|███████▉  | 238/300 [00:02<00:00, 75.62it/s]* GD progress: iter: 239 / bw: 6.028 / grad: -87.091 / stop criter: 6.856 :  79%|███████▉  | 238/300 [00:02<00:00, 75.62it/s]* GD progress: iter: 239 / bw: 6.028 / grad: -87.091 / stop criter: 6.856 :  79%|███████▉  | 238/300 [00:02<00:00, 75.62it/s]* GD progress: iter: 240 / bw: 6.122 / grad: 25.440 / stop criter: 87.091 :  79%|███████▉  | 238/300 [00:02<00:00, 75.62it/s]* GD progress: iter: 240 / bw: 6.122 / grad: 25.440 / stop criter: 87.091 :  79%|███████▉  | 238/300 [00:02<00:00, 75.62it/s]* GD progress: iter: 241 / bw: 6.095 / grad: 31.610 / stop criter: 25.440 :  79%|███████▉  | 238/300 [00:02<00:00, 75.62it/s]* GD progress: iter: 241 / bw: 6.095 / grad: 31.610 / stop criter: 25.440 :  79%|███████▉  | 238/300 [00:02<00:00, 75.62it/s]* GD progress: iter: 242 / bw: 6.061 / grad: -71.360 / stop criter: 31.610 :  79%|███████▉  | 238/300 [00:02<00:00, 75.62it/s]* GD progress: iter: 242 / bw: 6.061 / grad: -71.360 / stop criter: 31.610 :  79%|███████▉  | 238/300 [00:02<00:00, 75.62it/s]* GD progress: iter: 243 / bw: 6.137 / grad: 42.381 / stop criter: 71.360 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s] * GD progress: iter: 243 / bw: 6.137 / grad: 42.381 / stop criter: 71.360 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s]* GD progress: iter: 244 / bw: 6.092 / grad: 168.585 / stop criter: 42.381 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s]* GD progress: iter: 244 / bw: 6.092 / grad: 168.585 / stop criter: 42.381 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s]* GD progress: iter: 245 / bw: 5.917 / grad: 100.243 / stop criter: 168.585 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s]* GD progress: iter: 245 / bw: 5.917 / grad: 100.243 / stop criter: 168.585 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s]* GD progress: iter: 246 / bw: 5.813 / grad: -115.870 / stop criter: 100.243 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s]* GD progress: iter: 246 / bw: 5.813 / grad: -115.870 / stop criter: 100.243 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s]* GD progress: iter: 247 / bw: 5.933 / grad: -124.897 / stop criter: 115.870 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s]* GD progress: iter: 247 / bw: 5.933 / grad: -124.897 / stop criter: 115.870 :  79%|███████▉  | 238/300 [00:03<00:00, 75.62it/s]* GD progress: iter: 247 / bw: 5.933 / grad: -124.897 / stop criter: 115.870 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 248 / bw: 6.062 / grad: 20.357 / stop criter: 124.897 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]  * GD progress: iter: 248 / bw: 6.062 / grad: 20.357 / stop criter: 124.897 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 249 / bw: 6.041 / grad: 205.181 / stop criter: 20.357 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 249 / bw: 6.041 / grad: 205.181 / stop criter: 20.357 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 250 / bw: 5.832 / grad: -191.454 / stop criter: 205.181 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 250 / bw: 5.832 / grad: -191.454 / stop criter: 205.181 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 251 / bw: 6.027 / grad: 245.195 / stop criter: 191.454 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s] * GD progress: iter: 251 / bw: 6.027 / grad: 245.195 / stop criter: 191.454 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 252 / bw: 5.780 / grad: -25.546 / stop criter: 245.195 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 252 / bw: 5.780 / grad: -25.546 / stop criter: 245.195 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 253 / bw: 5.805 / grad: -302.118 / stop criter: 25.546 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 253 / bw: 5.805 / grad: -302.118 / stop criter: 25.546 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 254 / bw: 6.104 / grad: -2.182 / stop criter: 302.118 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s] * GD progress: iter: 254 / bw: 6.104 / grad: -2.182 / stop criter: 302.118 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 255 / bw: 6.106 / grad: 56.439 / stop criter: 2.182 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]  * GD progress: iter: 255 / bw: 6.106 / grad: 56.439 / stop criter: 2.182 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 256 / bw: 6.050 / grad: 91.487 / stop criter: 56.439 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 256 / bw: 6.050 / grad: 91.487 / stop criter: 56.439 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 257 / bw: 5.961 / grad: -46.561 / stop criter: 91.487 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 257 / bw: 5.961 / grad: -46.561 / stop criter: 91.487 :  82%|████████▏ | 247/300 [00:03<00:00, 77.36it/s]* GD progress: iter: 257 / bw: 5.961 / grad: -46.561 / stop criter: 91.487 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 258 / bw: 6.006 / grad: 28.323 / stop criter: 46.561 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s] * GD progress: iter: 258 / bw: 6.006 / grad: 28.323 / stop criter: 46.561 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 259 / bw: 5.979 / grad: -169.128 / stop criter: 28.323 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 259 / bw: 5.979 / grad: -169.128 / stop criter: 28.323 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 260 / bw: 6.141 / grad: 25.650 / stop criter: 169.128 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s] * GD progress: iter: 260 / bw: 6.141 / grad: 25.650 / stop criter: 169.128 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 261 / bw: 6.117 / grad: -49.348 / stop criter: 25.650 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 261 / bw: 6.117 / grad: -49.348 / stop criter: 25.650 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 262 / bw: 6.163 / grad: -20.992 / stop criter: 49.348 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 262 / bw: 6.163 / grad: -20.992 / stop criter: 49.348 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 263 / bw: 6.183 / grad: 28.480 / stop criter: 20.992 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s] * GD progress: iter: 263 / bw: 6.183 / grad: 28.480 / stop criter: 20.992 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 264 / bw: 6.156 / grad: -182.134 / stop criter: 28.480 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 264 / bw: 6.156 / grad: -182.134 / stop criter: 28.480 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 265 / bw: 6.324 / grad: -307.695 / stop criter: 182.134 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 265 / bw: 6.324 / grad: -307.695 / stop criter: 182.134 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 266 / bw: 6.605 / grad: 20.787 / stop criter: 307.695 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]  * GD progress: iter: 266 / bw: 6.605 / grad: 20.787 / stop criter: 307.695 :  86%|████████▌ | 257/300 [00:03<00:00, 80.49it/s]* GD progress: iter: 266 / bw: 6.605 / grad: 20.787 / stop criter: 307.695 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 267 / bw: 6.586 / grad: 34.771 / stop criter: 20.787 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s] * GD progress: iter: 267 / bw: 6.586 / grad: 34.771 / stop criter: 20.787 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 268 / bw: 6.554 / grad: 130.199 / stop criter: 34.771 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 268 / bw: 6.554 / grad: 130.199 / stop criter: 34.771 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 269 / bw: 6.437 / grad: -15.178 / stop criter: 130.199 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 269 / bw: 6.437 / grad: -15.178 / stop criter: 130.199 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 270 / bw: 6.450 / grad: -7.770 / stop criter: 15.178 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]  * GD progress: iter: 270 / bw: 6.450 / grad: -7.770 / stop criter: 15.178 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 271 / bw: 6.457 / grad: 81.926 / stop criter: 7.770 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s] * GD progress: iter: 271 / bw: 6.457 / grad: 81.926 / stop criter: 7.770 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 272 / bw: 6.384 / grad: -44.748 / stop criter: 81.926 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 272 / bw: 6.384 / grad: -44.748 / stop criter: 81.926 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 273 / bw: 6.424 / grad: 59.409 / stop criter: 44.748 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s] * GD progress: iter: 273 / bw: 6.424 / grad: 59.409 / stop criter: 44.748 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 274 / bw: 6.372 / grad: 6.971 / stop criter: 59.409 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s] * GD progress: iter: 274 / bw: 6.372 / grad: 6.971 / stop criter: 59.409 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 275 / bw: 6.366 / grad: 252.662 / stop criter: 6.971 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 275 / bw: 6.366 / grad: 252.662 / stop criter: 6.971 :  89%|████████▊ | 266/300 [00:03<00:00, 75.91it/s]* GD progress: iter: 275 / bw: 6.366 / grad: 252.662 / stop criter: 6.971 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 276 / bw: 6.146 / grad: -136.433 / stop criter: 252.662 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 276 / bw: 6.146 / grad: -136.433 / stop criter: 252.662 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 277 / bw: 6.265 / grad: 27.530 / stop criter: 136.433 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]  * GD progress: iter: 277 / bw: 6.265 / grad: 27.530 / stop criter: 136.433 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 278 / bw: 6.241 / grad: -194.486 / stop criter: 27.530 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 278 / bw: 6.241 / grad: -194.486 / stop criter: 27.530 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 279 / bw: 6.406 / grad: -3.730 / stop criter: 194.486 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s] * GD progress: iter: 279 / bw: 6.406 / grad: -3.730 / stop criter: 194.486 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 280 / bw: 6.410 / grad: 122.413 / stop criter: 3.730 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s] * GD progress: iter: 280 / bw: 6.410 / grad: 122.413 / stop criter: 3.730 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 281 / bw: 6.307 / grad: 185.867 / stop criter: 122.413 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 281 / bw: 6.307 / grad: 185.867 / stop criter: 122.413 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 282 / bw: 6.152 / grad: -80.013 / stop criter: 185.867 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 282 / bw: 6.152 / grad: -80.013 / stop criter: 185.867 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 283 / bw: 6.218 / grad: 102.451 / stop criter: 80.013 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s] * GD progress: iter: 283 / bw: 6.218 / grad: 102.451 / stop criter: 80.013 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 284 / bw: 6.134 / grad: 63.982 / stop criter: 102.451 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 284 / bw: 6.134 / grad: 63.982 / stop criter: 102.451 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 285 / bw: 6.081 / grad: -142.696 / stop criter: 63.982 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 285 / bw: 6.081 / grad: -142.696 / stop criter: 63.982 :  92%|█████████▏| 275/300 [00:03<00:00, 79.25it/s]* GD progress: iter: 285 / bw: 6.081 / grad: -142.696 / stop criter: 63.982 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 286 / bw: 6.198 / grad: -153.223 / stop criter: 142.696 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 286 / bw: 6.198 / grad: -153.223 / stop criter: 142.696 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 287 / bw: 6.322 / grad: -22.068 / stop criter: 153.223 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s] * GD progress: iter: 287 / bw: 6.322 / grad: -22.068 / stop criter: 153.223 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 288 / bw: 6.340 / grad: 217.453 / stop criter: 22.068 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s] * GD progress: iter: 288 / bw: 6.340 / grad: 217.453 / stop criter: 22.068 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 289 / bw: 6.164 / grad: -194.913 / stop criter: 217.453 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 289 / bw: 6.164 / grad: -194.913 / stop criter: 217.453 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 290 / bw: 6.320 / grad: -19.462 / stop criter: 194.913 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s] * GD progress: iter: 290 / bw: 6.320 / grad: -19.462 / stop criter: 194.913 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 291 / bw: 6.335 / grad: 139.137 / stop criter: 19.462 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s] * GD progress: iter: 291 / bw: 6.335 / grad: 139.137 / stop criter: 19.462 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 292 / bw: 6.225 / grad: 176.398 / stop criter: 139.137 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 292 / bw: 6.225 / grad: 176.398 / stop criter: 139.137 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 293 / bw: 6.087 / grad: 9.538 / stop criter: 176.398 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]  * GD progress: iter: 293 / bw: 6.087 / grad: 9.538 / stop criter: 176.398 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 294 / bw: 6.079 / grad: 1.077 / stop criter: 9.538 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]  * GD progress: iter: 294 / bw: 6.079 / grad: 1.077 / stop criter: 9.538 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 295 / bw: 6.078 / grad: 96.985 / stop criter: 1.077 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 295 / bw: 6.078 / grad: 96.985 / stop criter: 1.077 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 296 / bw: 6.002 / grad: 10.792 / stop criter: 96.985 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 296 / bw: 6.002 / grad: 10.792 / stop criter: 96.985 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 297 / bw: 5.994 / grad: 78.838 / stop criter: 10.792 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 297 / bw: 5.994 / grad: 78.838 / stop criter: 10.792 :  95%|█████████▌| 285/300 [00:03<00:00, 81.23it/s]* GD progress: iter: 297 / bw: 5.994 / grad: 78.838 / stop criter: 10.792 :  99%|█████████▉| 297/300 [00:03<00:00, 87.63it/s]* GD progress: iter: 298 / bw: 5.932 / grad: -17.450 / stop criter: 78.838 :  99%|█████████▉| 297/300 [00:03<00:00, 87.63it/s]* GD progress: iter: 298 / bw: 5.932 / grad: -17.450 / stop criter: 78.838 :  99%|█████████▉| 297/300 [00:03<00:00, 87.63it/s]* GD progress: iter: 299 / bw: 5.946 / grad: 35.293 / stop criter: 17.450 :  99%|█████████▉| 297/300 [00:03<00:00, 87.63it/s] * GD progress: iter: 299 / bw: 5.946 / grad: 35.293 / stop criter: 17.450 :  99%|█████████▉| 297/300 [00:03<00:00, 87.63it/s]* GD progress: iter: 300 / bw: 5.918 / grad: -86.568 / stop criter: 35.293 :  99%|█████████▉| 297/300 [00:03<00:00, 87.63it/s]* GD progress: iter: 300 / bw: 5.918 / grad: -86.568 / stop criter: 35.293 :  99%|█████████▉| 297/300 [00:03<00:00, 87.63it/s]* GD progress: iter: 300 / bw: 5.918 / grad: -86.568 / stop criter: 35.293 : 100%|██████████| 300/300 [00:03<00:00, 81.89it/s]

The estimated optimal bandwidth is given by gc1.optimization_outputs['opt_bandwidth'].

# Gradient COBRA with default parameter
print("Estimated bandwidth :" + str(gc1_fit.optimization_outputs['opt_bandwidth']))

Estimated bandwidth :5.9182619785634

We can look at the learning curve of the algorithm using draw_learning_curve() method.

gc1_fit.draw_learning_curve()

We evaluate the performance of the method on the testing data using MSE and MAPE.

from sklearn.metrics import mean_absolute_percentage_error
y_pred1 = gc1_fit.predict(X_test1)
print(mean_absolute_percentage_error(y_test1, y_pred1))
print(mean_squared_error(y_test1, y_pred1))

0.13639409238371467
77.10625607904514

Let’s look at qq-plot of the predictions and the actual response values using draw_learning_curve() method.

gc1_fit.draw_learning_curve(y_test=y_test1, fig_type='qq')

\(\bullet\) `GradientCOBRA` with non-default parameters

We create another object gc2 with non-default parameters, then fit it to the same training data as in the previous example.

gc2 = GradientCOBRA(learning_rate=0.01,
                    opt_method='grad',
                    loss_function="weighted_mse",
                    max_iter=300,
                    estimator_list=['random_forest', 
                                    'adaboost', 
                                    'knn', 
                                    'lasso', 
                                    'ridge'],
                    estimator_params={
                        'random_forest' : {
                            'n_estimators' : 300,
                            'min_samples_leaf' : 5},

                        'adaboost' : {
                            'n_estimators' : 300,
                            'max_depth' : 5},
                        
                        'knn' : {
                            'n_neighbors' : 10}
                    })

gc2_fit = gc2.fit(X_train1, y_train1)

* GD progress: iter: 1 / bw: 3.000 / grad: -7.017 / stop criter: 1.000 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 1 / bw: 3.010 / grad: -7.017 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 1 / bw: 3.010 / grad: -7.017 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 2 / bw: 3.010 / grad: -7.017 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 2 / bw: 3.010 / grad: -7.017 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 3 / bw: 3.010 / grad: -7.017 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 3 / bw: 3.010 / grad: -7.017 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 4 / bw: 3.010 / grad: -7.017 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 4 / bw: 3.010 / grad: -7.017 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 5 / bw: 3.010 / grad: -9.084 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 5 / bw: 3.010 / grad: -9.084 / stop criter: 7.017 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 6 / bw: 3.023 / grad: -42.455 / stop criter: 9.084 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 6 / bw: 3.023 / grad: -42.455 / stop criter: 9.084 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 7 / bw: 3.083 / grad: -33.491 / stop criter: 42.455 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 7 / bw: 3.083 / grad: -33.491 / stop criter: 42.455 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 8 / bw: 3.131 / grad: -5.657 / stop criter: 33.491 :   0%|          | 0/300 [00:00<?, ?it/s] * GD progress: iter: 8 / bw: 3.131 / grad: -5.657 / stop criter: 33.491 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 9 / bw: 3.139 / grad: -13.805 / stop criter: 5.657 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 9 / bw: 3.139 / grad: -13.805 / stop criter: 5.657 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 10 / bw: 3.159 / grad: -11.947 / stop criter: 13.805 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 10 / bw: 3.159 / grad: -11.947 / stop criter: 13.805 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 11 / bw: 3.176 / grad: 27.805 / stop criter: 11.947 :   0%|          | 0/300 [00:00<?, ?it/s] * GD progress: iter: 11 / bw: 3.176 / grad: 27.805 / stop criter: 11.947 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 11 / bw: 3.176 / grad: 27.805 / stop criter: 11.947 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 12 / bw: 3.136 / grad: -6.482 / stop criter: 27.805 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 12 / bw: 3.136 / grad: -6.482 / stop criter: 27.805 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 13 / bw: 3.145 / grad: 20.481 / stop criter: 6.482 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s] * GD progress: iter: 13 / bw: 3.145 / grad: 20.481 / stop criter: 6.482 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 14 / bw: 3.117 / grad: -0.323 / stop criter: 20.481 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 14 / bw: 3.117 / grad: -0.323 / stop criter: 20.481 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 15 / bw: 3.117 / grad: 28.905 / stop criter: 0.323 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s] * GD progress: iter: 15 / bw: 3.117 / grad: 28.905 / stop criter: 0.323 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 16 / bw: 3.078 / grad: 23.724 / stop criter: 28.905 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 16 / bw: 3.078 / grad: 23.724 / stop criter: 28.905 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 17 / bw: 3.046 / grad: -4.997 / stop criter: 23.724 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 17 / bw: 3.046 / grad: -4.997 / stop criter: 23.724 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 18 / bw: 3.052 / grad: 28.048 / stop criter: 4.997 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s] * GD progress: iter: 18 / bw: 3.052 / grad: 28.048 / stop criter: 4.997 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 19 / bw: 3.015 / grad: 0.859 / stop criter: 28.048 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 19 / bw: 3.015 / grad: 0.859 / stop criter: 28.048 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 20 / bw: 3.014 / grad: -39.115 / stop criter: 0.859 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 20 / bw: 3.014 / grad: -39.115 / stop criter: 0.859 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 21 / bw: 3.066 / grad: 30.672 / stop criter: 39.115 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 21 / bw: 3.066 / grad: 30.672 / stop criter: 39.115 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 22 / bw: 3.025 / grad: 29.997 / stop criter: 30.672 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 22 / bw: 3.025 / grad: 29.997 / stop criter: 30.672 :   4%|▎         | 11/300 [00:00<00:02, 100.01it/s]* GD progress: iter: 22 / bw: 3.025 / grad: 29.997 / stop criter: 30.672 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 23 / bw: 2.986 / grad: -30.430 / stop criter: 29.997 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 23 / bw: 2.986 / grad: -30.430 / stop criter: 29.997 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 24 / bw: 3.026 / grad: 25.623 / stop criter: 30.430 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s] * GD progress: iter: 24 / bw: 3.026 / grad: 25.623 / stop criter: 30.430 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 25 / bw: 2.993 / grad: -3.610 / stop criter: 25.623 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 25 / bw: 2.993 / grad: -3.610 / stop criter: 25.623 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 26 / bw: 2.997 / grad: -4.356 / stop criter: 3.610 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s] * GD progress: iter: 26 / bw: 2.997 / grad: -4.356 / stop criter: 3.610 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 27 / bw: 3.003 / grad: -5.057 / stop criter: 4.356 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 27 / bw: 3.003 / grad: -5.057 / stop criter: 4.356 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 28 / bw: 3.009 / grad: -41.575 / stop criter: 5.057 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 28 / bw: 3.009 / grad: -41.575 / stop criter: 5.057 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 29 / bw: 3.062 / grad: -2.358 / stop criter: 41.575 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 29 / bw: 3.062 / grad: -2.358 / stop criter: 41.575 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 30 / bw: 3.065 / grad: -0.162 / stop criter: 2.358 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s] * GD progress: iter: 30 / bw: 3.065 / grad: -0.162 / stop criter: 2.358 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 31 / bw: 3.065 / grad: -8.812 / stop criter: 0.162 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 31 / bw: 3.065 / grad: -8.812 / stop criter: 0.162 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 32 / bw: 3.076 / grad: -10.666 / stop criter: 8.812 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 32 / bw: 3.076 / grad: -10.666 / stop criter: 8.812 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 33 / bw: 3.090 / grad: 0.177 / stop criter: 10.666 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s] * GD progress: iter: 33 / bw: 3.090 / grad: 0.177 / stop criter: 10.666 :   7%|▋         | 22/300 [00:00<00:02, 102.29it/s]* GD progress: iter: 33 / bw: 3.090 / grad: 0.177 / stop criter: 10.666 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 34 / bw: 3.089 / grad: 28.312 / stop criter: 0.177 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 34 / bw: 3.089 / grad: 28.312 / stop criter: 0.177 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 35 / bw: 3.054 / grad: -6.119 / stop criter: 28.312 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 35 / bw: 3.054 / grad: -6.119 / stop criter: 28.312 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 36 / bw: 3.062 / grad: -15.112 / stop criter: 6.119 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 36 / bw: 3.062 / grad: -15.112 / stop criter: 6.119 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 37 / bw: 3.080 / grad: -8.363 / stop criter: 15.112 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 37 / bw: 3.080 / grad: -8.363 / stop criter: 15.112 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 38 / bw: 3.091 / grad: 3.158 / stop criter: 8.363 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]  * GD progress: iter: 38 / bw: 3.091 / grad: 3.158 / stop criter: 8.363 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 39 / bw: 3.087 / grad: 2.412 / stop criter: 3.158 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 39 / bw: 3.087 / grad: 2.412 / stop criter: 3.158 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 40 / bw: 3.084 / grad: -2.519 / stop criter: 2.412 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 40 / bw: 3.084 / grad: -2.519 / stop criter: 2.412 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 41 / bw: 3.087 / grad: -39.742 / stop criter: 2.519 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 41 / bw: 3.087 / grad: -39.742 / stop criter: 2.519 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 42 / bw: 3.135 / grad: -8.270 / stop criter: 39.742 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 42 / bw: 3.135 / grad: -8.270 / stop criter: 39.742 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 43 / bw: 3.145 / grad: -25.263 / stop criter: 8.270 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 43 / bw: 3.145 / grad: -25.263 / stop criter: 8.270 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 44 / bw: 3.176 / grad: 8.485 / stop criter: 25.263 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s] * GD progress: iter: 44 / bw: 3.176 / grad: 8.485 / stop criter: 25.263 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 45 / bw: 3.165 / grad: -20.039 / stop criter: 8.485 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 45 / bw: 3.165 / grad: -20.039 / stop criter: 8.485 :  11%|█         | 33/300 [00:00<00:02, 105.48it/s]* GD progress: iter: 45 / bw: 3.165 / grad: -20.039 / stop criter: 8.485 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 46 / bw: 3.190 / grad: -8.670 / stop criter: 20.039 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 46 / bw: 3.190 / grad: -8.670 / stop criter: 20.039 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 47 / bw: 3.200 / grad: -6.997 / stop criter: 8.670 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s] * GD progress: iter: 47 / bw: 3.200 / grad: -6.997 / stop criter: 8.670 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 48 / bw: 3.208 / grad: -21.076 / stop criter: 6.997 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 48 / bw: 3.208 / grad: -21.076 / stop criter: 6.997 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 49 / bw: 3.233 / grad: -3.583 / stop criter: 21.076 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 49 / bw: 3.233 / grad: -3.583 / stop criter: 21.076 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 50 / bw: 3.237 / grad: -23.114 / stop criter: 3.583 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 50 / bw: 3.237 / grad: -23.114 / stop criter: 3.583 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 51 / bw: 3.265 / grad: -5.918 / stop criter: 23.114 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 51 / bw: 3.265 / grad: -5.918 / stop criter: 23.114 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 52 / bw: 3.272 / grad: -31.179 / stop criter: 5.918 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 52 / bw: 3.272 / grad: -31.179 / stop criter: 5.918 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 53 / bw: 3.309 / grad: -5.204 / stop criter: 31.179 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 53 / bw: 3.309 / grad: -5.204 / stop criter: 31.179 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 54 / bw: 3.315 / grad: 28.047 / stop criter: 5.204 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s] * GD progress: iter: 54 / bw: 3.315 / grad: 28.047 / stop criter: 5.204 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 55 / bw: 3.282 / grad: -25.611 / stop criter: 28.047 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 55 / bw: 3.282 / grad: -25.611 / stop criter: 28.047 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 56 / bw: 3.312 / grad: -14.387 / stop criter: 25.611 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 56 / bw: 3.312 / grad: -14.387 / stop criter: 25.611 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 57 / bw: 3.329 / grad: 10.252 / stop criter: 14.387 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s] * GD progress: iter: 57 / bw: 3.329 / grad: 10.252 / stop criter: 14.387 :  15%|█▌        | 45/300 [00:00<00:02, 104.29it/s]* GD progress: iter: 57 / bw: 3.329 / grad: 10.252 / stop criter: 14.387 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 58 / bw: 3.317 / grad: 18.526 / stop criter: 10.252 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 58 / bw: 3.317 / grad: 18.526 / stop criter: 10.252 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 59 / bw: 3.296 / grad: 2.517 / stop criter: 18.526 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s] * GD progress: iter: 59 / bw: 3.296 / grad: 2.517 / stop criter: 18.526 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 60 / bw: 3.293 / grad: -21.964 / stop criter: 2.517 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 60 / bw: 3.293 / grad: -21.964 / stop criter: 2.517 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 61 / bw: 3.318 / grad: -2.948 / stop criter: 21.964 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 61 / bw: 3.318 / grad: -2.948 / stop criter: 21.964 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 62 / bw: 3.321 / grad: -34.176 / stop criter: 2.948 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 62 / bw: 3.321 / grad: -34.176 / stop criter: 2.948 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 63 / bw: 3.360 / grad: -14.301 / stop criter: 34.176 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 63 / bw: 3.360 / grad: -14.301 / stop criter: 34.176 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 64 / bw: 3.377 / grad: 9.090 / stop criter: 14.301 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]  * GD progress: iter: 64 / bw: 3.377 / grad: 9.090 / stop criter: 14.301 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 65 / bw: 3.366 / grad: -46.398 / stop criter: 9.090 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 65 / bw: 3.366 / grad: -46.398 / stop criter: 9.090 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 66 / bw: 3.419 / grad: 14.528 / stop criter: 46.398 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 66 / bw: 3.419 / grad: 14.528 / stop criter: 46.398 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 67 / bw: 3.403 / grad: 17.132 / stop criter: 14.528 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 67 / bw: 3.403 / grad: 17.132 / stop criter: 14.528 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 68 / bw: 3.384 / grad: 6.730 / stop criter: 17.132 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s] * GD progress: iter: 68 / bw: 3.384 / grad: 6.730 / stop criter: 17.132 :  19%|█▉        | 57/300 [00:00<00:02, 103.65it/s]* GD progress: iter: 68 / bw: 3.384 / grad: 6.730 / stop criter: 17.132 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 69 / bw: 3.376 / grad: 19.486 / stop criter: 6.730 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 69 / bw: 3.376 / grad: 19.486 / stop criter: 6.730 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 70 / bw: 3.354 / grad: 9.439 / stop criter: 19.486 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 70 / bw: 3.354 / grad: 9.439 / stop criter: 19.486 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 71 / bw: 3.344 / grad: -33.204 / stop criter: 9.439 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 71 / bw: 3.344 / grad: -33.204 / stop criter: 9.439 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 72 / bw: 3.381 / grad: 8.229 / stop criter: 33.204 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s] * GD progress: iter: 72 / bw: 3.381 / grad: 8.229 / stop criter: 33.204 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 73 / bw: 3.372 / grad: -32.752 / stop criter: 8.229 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 73 / bw: 3.372 / grad: -32.752 / stop criter: 8.229 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 74 / bw: 3.407 / grad: -6.111 / stop criter: 32.752 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 74 / bw: 3.407 / grad: -6.111 / stop criter: 32.752 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 75 / bw: 3.414 / grad: -8.402 / stop criter: 6.111 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s] * GD progress: iter: 75 / bw: 3.414 / grad: -8.402 / stop criter: 6.111 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 76 / bw: 3.423 / grad: -8.667 / stop criter: 8.402 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 76 / bw: 3.423 / grad: -8.667 / stop criter: 8.402 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 77 / bw: 3.432 / grad: 6.062 / stop criter: 8.667 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s] * GD progress: iter: 77 / bw: 3.432 / grad: 6.062 / stop criter: 8.667 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 78 / bw: 3.426 / grad: -11.075 / stop criter: 6.062 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 78 / bw: 3.426 / grad: -11.075 / stop criter: 6.062 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 79 / bw: 3.438 / grad: -24.418 / stop criter: 11.075 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 79 / bw: 3.438 / grad: -24.418 / stop criter: 11.075 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 80 / bw: 3.463 / grad: 13.203 / stop criter: 24.418 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s] * GD progress: iter: 80 / bw: 3.463 / grad: 13.203 / stop criter: 24.418 :  23%|██▎       | 68/300 [00:00<00:02, 105.45it/s]* GD progress: iter: 80 / bw: 3.463 / grad: 13.203 / stop criter: 24.418 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 81 / bw: 3.449 / grad: 20.319 / stop criter: 13.203 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 81 / bw: 3.449 / grad: 20.319 / stop criter: 13.203 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 82 / bw: 3.428 / grad: -30.161 / stop criter: 20.319 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 82 / bw: 3.428 / grad: -30.161 / stop criter: 20.319 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 83 / bw: 3.460 / grad: -6.248 / stop criter: 30.161 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s] * GD progress: iter: 83 / bw: 3.460 / grad: -6.248 / stop criter: 30.161 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 84 / bw: 3.466 / grad: -10.664 / stop criter: 6.248 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 84 / bw: 3.466 / grad: -10.664 / stop criter: 6.248 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 85 / bw: 3.477 / grad: 9.916 / stop criter: 10.664 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s] * GD progress: iter: 85 / bw: 3.477 / grad: 9.916 / stop criter: 10.664 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 86 / bw: 3.467 / grad: 20.695 / stop criter: 9.916 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 86 / bw: 3.467 / grad: 20.695 / stop criter: 9.916 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 87 / bw: 3.446 / grad: 43.095 / stop criter: 20.695 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 87 / bw: 3.446 / grad: 43.095 / stop criter: 20.695 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 88 / bw: 3.402 / grad: 16.572 / stop criter: 43.095 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 88 / bw: 3.402 / grad: 16.572 / stop criter: 43.095 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 89 / bw: 3.385 / grad: 12.916 / stop criter: 16.572 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 89 / bw: 3.385 / grad: 12.916 / stop criter: 16.572 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 90 / bw: 3.371 / grad: 14.709 / stop criter: 12.916 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 90 / bw: 3.371 / grad: 14.709 / stop criter: 12.916 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 91 / bw: 3.356 / grad: 31.404 / stop criter: 14.709 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 91 / bw: 3.356 / grad: 31.404 / stop criter: 14.709 :  27%|██▋       | 80/300 [00:00<00:02, 104.51it/s]* GD progress: iter: 91 / bw: 3.356 / grad: 31.404 / stop criter: 14.709 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 92 / bw: 3.324 / grad: -3.768 / stop criter: 31.404 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 92 / bw: 3.324 / grad: -3.768 / stop criter: 31.404 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 93 / bw: 3.328 / grad: 20.555 / stop criter: 3.768 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s] * GD progress: iter: 93 / bw: 3.328 / grad: 20.555 / stop criter: 3.768 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 94 / bw: 3.307 / grad: -49.996 / stop criter: 20.555 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 94 / bw: 3.307 / grad: -49.996 / stop criter: 20.555 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 95 / bw: 3.357 / grad: 12.857 / stop criter: 49.996 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s] * GD progress: iter: 95 / bw: 3.357 / grad: 12.857 / stop criter: 49.996 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 96 / bw: 3.345 / grad: -11.207 / stop criter: 12.857 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 96 / bw: 3.345 / grad: -11.207 / stop criter: 12.857 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 97 / bw: 3.356 / grad: 23.186 / stop criter: 11.207 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s] * GD progress: iter: 97 / bw: 3.356 / grad: 23.186 / stop criter: 11.207 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 98 / bw: 3.333 / grad: -5.652 / stop criter: 23.186 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 98 / bw: 3.333 / grad: -5.652 / stop criter: 23.186 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 99 / bw: 3.339 / grad: 36.205 / stop criter: 5.652 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s] * GD progress: iter: 99 / bw: 3.339 / grad: 36.205 / stop criter: 5.652 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 100 / bw: 3.304 / grad: 57.925 / stop criter: 36.205 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 100 / bw: 3.304 / grad: 57.925 / stop criter: 36.205 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 101 / bw: 3.249 / grad: -22.059 / stop criter: 57.925 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 101 / bw: 3.249 / grad: -22.059 / stop criter: 57.925 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 102 / bw: 3.270 / grad: 13.486 / stop criter: 22.059 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s] * GD progress: iter: 102 / bw: 3.270 / grad: 13.486 / stop criter: 22.059 :  30%|███       | 91/300 [00:00<00:01, 105.97it/s]* GD progress: iter: 102 / bw: 3.270 / grad: 13.486 / stop criter: 22.059 :  34%|███▍      | 102/300 [00:00<00:01, 104.50it/s]* GD progress: iter: 103 / bw: 3.258 / grad: 8.508 / stop criter: 13.486 :  34%|███▍      | 102/300 [00:00<00:01, 104.50it/s] * GD progress: iter: 103 / bw: 3.258 / grad: 8.508 / stop criter: 13.486 :  34%|███▍      | 102/300 [00:00<00:01, 104.50it/s]* GD progress: iter: 104 / bw: 3.250 / grad: -45.509 / stop criter: 8.508 :  34%|███▍      | 102/300 [00:00<00:01, 104.50it/s]* GD progress: iter: 104 / bw: 3.250 / grad: -45.509 / stop criter: 8.508 :  34%|███▍      | 102/300 [00:00<00:01, 104.50it/s]* GD progress: iter: 105 / bw: 3.292 / grad: 12.331 / stop criter: 45.509 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 105 / bw: 3.292 / grad: 12.331 / stop criter: 45.509 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 106 / bw: 3.281 / grad: 13.146 / stop criter: 12.331 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 106 / bw: 3.281 / grad: 13.146 / stop criter: 12.331 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 107 / bw: 3.269 / grad: 11.597 / stop criter: 13.146 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 107 / bw: 3.269 / grad: 11.597 / stop criter: 13.146 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 108 / bw: 3.258 / grad: 16.450 / stop criter: 11.597 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 108 / bw: 3.258 / grad: 16.450 / stop criter: 11.597 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 109 / bw: 3.243 / grad: 17.020 / stop criter: 16.450 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 109 / bw: 3.243 / grad: 17.020 / stop criter: 16.450 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 110 / bw: 3.228 / grad: 20.469 / stop criter: 17.020 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 110 / bw: 3.228 / grad: 20.469 / stop criter: 17.020 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 111 / bw: 3.209 / grad: 20.619 / stop criter: 20.469 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 111 / bw: 3.209 / grad: 20.619 / stop criter: 20.469 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 112 / bw: 3.191 / grad: -0.731 / stop criter: 20.619 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 112 / bw: 3.191 / grad: -0.731 / stop criter: 20.619 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 113 / bw: 3.191 / grad: 10.937 / stop criter: 0.731 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s] * GD progress: iter: 113 / bw: 3.191 / grad: 10.937 / stop criter: 0.731 :  34%|███▍      | 102/300 [00:01<00:01, 104.50it/s]* GD progress: iter: 113 / bw: 3.191 / grad: 10.937 / stop criter: 0.731 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 114 / bw: 3.181 / grad: 13.007 / stop criter: 10.937 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 114 / bw: 3.181 / grad: 13.007 / stop criter: 10.937 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 115 / bw: 3.170 / grad: -4.850 / stop criter: 13.007 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 115 / bw: 3.170 / grad: -4.850 / stop criter: 13.007 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 116 / bw: 3.174 / grad: 18.558 / stop criter: 4.850 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s] * GD progress: iter: 116 / bw: 3.174 / grad: 18.558 / stop criter: 4.850 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 117 / bw: 3.158 / grad: 14.681 / stop criter: 18.558 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 117 / bw: 3.158 / grad: 14.681 / stop criter: 18.558 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 118 / bw: 3.145 / grad: 28.390 / stop criter: 14.681 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 118 / bw: 3.145 / grad: 28.390 / stop criter: 14.681 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 119 / bw: 3.120 / grad: -5.251 / stop criter: 28.390 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 119 / bw: 3.120 / grad: -5.251 / stop criter: 28.390 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 120 / bw: 3.125 / grad: -3.583 / stop criter: 5.251 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s] * GD progress: iter: 120 / bw: 3.125 / grad: -3.583 / stop criter: 5.251 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 121 / bw: 3.128 / grad: 30.809 / stop criter: 3.583 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 121 / bw: 3.128 / grad: 30.809 / stop criter: 3.583 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 122 / bw: 3.101 / grad: 1.394 / stop criter: 30.809 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 122 / bw: 3.101 / grad: 1.394 / stop criter: 30.809 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 123 / bw: 3.100 / grad: -11.548 / stop criter: 1.394 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 123 / bw: 3.100 / grad: -11.548 / stop criter: 1.394 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 124 / bw: 3.110 / grad: 54.697 / stop criter: 11.548 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 124 / bw: 3.110 / grad: 54.697 / stop criter: 11.548 :  38%|███▊      | 113/300 [00:01<00:01, 103.89it/s]* GD progress: iter: 124 / bw: 3.110 / grad: 54.697 / stop criter: 11.548 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 125 / bw: 3.064 / grad: 32.108 / stop criter: 54.697 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 125 / bw: 3.064 / grad: 32.108 / stop criter: 54.697 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 126 / bw: 3.037 / grad: -6.650 / stop criter: 32.108 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 126 / bw: 3.037 / grad: -6.650 / stop criter: 32.108 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 127 / bw: 3.043 / grad: -10.907 / stop criter: 6.650 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 127 / bw: 3.043 / grad: -10.907 / stop criter: 6.650 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 128 / bw: 3.052 / grad: -1.024 / stop criter: 10.907 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 128 / bw: 3.052 / grad: -1.024 / stop criter: 10.907 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 129 / bw: 3.052 / grad: -44.380 / stop criter: 1.024 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 129 / bw: 3.052 / grad: -44.380 / stop criter: 1.024 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 130 / bw: 3.089 / grad: 20.802 / stop criter: 44.380 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 130 / bw: 3.089 / grad: 20.802 / stop criter: 44.380 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 131 / bw: 3.072 / grad: -3.944 / stop criter: 20.802 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 131 / bw: 3.072 / grad: -3.944 / stop criter: 20.802 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 132 / bw: 3.075 / grad: -30.437 / stop criter: 3.944 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 132 / bw: 3.075 / grad: -30.437 / stop criter: 3.944 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 133 / bw: 3.100 / grad: -3.580 / stop criter: 30.437 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 133 / bw: 3.100 / grad: -3.580 / stop criter: 30.437 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 134 / bw: 3.103 / grad: 1.697 / stop criter: 3.580 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]  * GD progress: iter: 134 / bw: 3.103 / grad: 1.697 / stop criter: 3.580 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 135 / bw: 3.101 / grad: -0.944 / stop criter: 1.697 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 135 / bw: 3.101 / grad: -0.944 / stop criter: 1.697 :  41%|████▏     | 124/300 [00:01<00:01, 103.40it/s]* GD progress: iter: 135 / bw: 3.101 / grad: -0.944 / stop criter: 1.697 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 136 / bw: 3.102 / grad: 0.979 / stop criter: 0.944 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s] * GD progress: iter: 136 / bw: 3.102 / grad: 0.979 / stop criter: 0.944 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 137 / bw: 3.101 / grad: -7.335 / stop criter: 0.979 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 137 / bw: 3.101 / grad: -7.335 / stop criter: 0.979 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 138 / bw: 3.107 / grad: -4.323 / stop criter: 7.335 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 138 / bw: 3.107 / grad: -4.323 / stop criter: 7.335 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 139 / bw: 3.111 / grad: 1.010 / stop criter: 4.323 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s] * GD progress: iter: 139 / bw: 3.111 / grad: 1.010 / stop criter: 4.323 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 140 / bw: 3.110 / grad: 57.737 / stop criter: 1.010 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 140 / bw: 3.110 / grad: 57.737 / stop criter: 1.010 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 141 / bw: 3.065 / grad: -11.760 / stop criter: 57.737 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 141 / bw: 3.065 / grad: -11.760 / stop criter: 57.737 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 142 / bw: 3.074 / grad: 61.032 / stop criter: 11.760 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s] * GD progress: iter: 142 / bw: 3.074 / grad: 61.032 / stop criter: 11.760 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 143 / bw: 3.028 / grad: 24.830 / stop criter: 61.032 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 143 / bw: 3.028 / grad: 24.830 / stop criter: 61.032 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 144 / bw: 3.009 / grad: -4.566 / stop criter: 24.830 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 144 / bw: 3.009 / grad: -4.566 / stop criter: 24.830 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 145 / bw: 3.012 / grad: -1.758 / stop criter: 4.566 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s] * GD progress: iter: 145 / bw: 3.012 / grad: -1.758 / stop criter: 4.566 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 146 / bw: 3.014 / grad: -8.718 / stop criter: 1.758 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 146 / bw: 3.014 / grad: -8.718 / stop criter: 1.758 :  45%|████▌     | 135/300 [00:01<00:01, 102.56it/s]* GD progress: iter: 146 / bw: 3.014 / grad: -8.718 / stop criter: 1.758 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 147 / bw: 3.020 / grad: 27.930 / stop criter: 8.718 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 147 / bw: 3.020 / grad: 27.930 / stop criter: 8.718 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 148 / bw: 2.999 / grad: 26.889 / stop criter: 27.930 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 148 / bw: 2.999 / grad: 26.889 / stop criter: 27.930 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 149 / bw: 2.979 / grad: 6.072 / stop criter: 26.889 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s] * GD progress: iter: 149 / bw: 2.979 / grad: 6.072 / stop criter: 26.889 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 150 / bw: 2.975 / grad: 26.797 / stop criter: 6.072 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 150 / bw: 2.975 / grad: 26.797 / stop criter: 6.072 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 151 / bw: 2.955 / grad: 25.936 / stop criter: 26.797 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 151 / bw: 2.955 / grad: 25.936 / stop criter: 26.797 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 152 / bw: 2.936 / grad: -10.363 / stop criter: 25.936 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 152 / bw: 2.936 / grad: -10.363 / stop criter: 25.936 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 153 / bw: 2.943 / grad: -33.350 / stop criter: 10.363 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 153 / bw: 2.943 / grad: -33.350 / stop criter: 10.363 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 154 / bw: 2.968 / grad: -6.745 / stop criter: 33.350 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s] * GD progress: iter: 154 / bw: 2.968 / grad: -6.745 / stop criter: 33.350 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 155 / bw: 2.973 / grad: -13.657 / stop criter: 6.745 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 155 / bw: 2.973 / grad: -13.657 / stop criter: 6.745 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 156 / bw: 2.983 / grad: -8.670 / stop criter: 13.657 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 156 / bw: 2.983 / grad: -8.670 / stop criter: 13.657 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 157 / bw: 2.989 / grad: 59.700 / stop criter: 8.670 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s] * GD progress: iter: 157 / bw: 2.989 / grad: 59.700 / stop criter: 8.670 :  49%|████▊     | 146/300 [00:01<00:01, 102.15it/s]* GD progress: iter: 157 / bw: 2.989 / grad: 59.700 / stop criter: 8.670 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 158 / bw: 2.945 / grad: -7.384 / stop criter: 59.700 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 158 / bw: 2.945 / grad: -7.384 / stop criter: 59.700 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 159 / bw: 2.951 / grad: 23.538 / stop criter: 7.384 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s] * GD progress: iter: 159 / bw: 2.951 / grad: 23.538 / stop criter: 7.384 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 160 / bw: 2.934 / grad: -5.513 / stop criter: 23.538 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 160 / bw: 2.934 / grad: -5.513 / stop criter: 23.538 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 161 / bw: 2.938 / grad: -6.826 / stop criter: 5.513 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s] * GD progress: iter: 161 / bw: 2.938 / grad: -6.826 / stop criter: 5.513 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 162 / bw: 2.943 / grad: -3.844 / stop criter: 6.826 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 162 / bw: 2.943 / grad: -3.844 / stop criter: 6.826 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 163 / bw: 2.945 / grad: -0.445 / stop criter: 3.844 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 163 / bw: 2.945 / grad: -0.445 / stop criter: 3.844 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 164 / bw: 2.946 / grad: -14.734 / stop criter: 0.445 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 164 / bw: 2.946 / grad: -14.734 / stop criter: 0.445 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 165 / bw: 2.956 / grad: -7.493 / stop criter: 14.734 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 165 / bw: 2.956 / grad: -7.493 / stop criter: 14.734 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 166 / bw: 2.961 / grad: 57.409 / stop criter: 7.493 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s] * GD progress: iter: 166 / bw: 2.961 / grad: 57.409 / stop criter: 7.493 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 167 / bw: 2.921 / grad: -37.853 / stop criter: 57.409 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 167 / bw: 2.921 / grad: -37.853 / stop criter: 57.409 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 168 / bw: 2.947 / grad: -3.525 / stop criter: 37.853 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s] * GD progress: iter: 168 / bw: 2.947 / grad: -3.525 / stop criter: 37.853 :  52%|█████▏    | 157/300 [00:01<00:01, 102.40it/s]* GD progress: iter: 168 / bw: 2.947 / grad: -3.525 / stop criter: 37.853 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 169 / bw: 2.950 / grad: -6.037 / stop criter: 3.525 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s] * GD progress: iter: 169 / bw: 2.950 / grad: -6.037 / stop criter: 3.525 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 170 / bw: 2.954 / grad: -6.144 / stop criter: 6.037 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 170 / bw: 2.954 / grad: -6.144 / stop criter: 6.037 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 171 / bw: 2.958 / grad: 2.829 / stop criter: 6.144 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s] * GD progress: iter: 171 / bw: 2.958 / grad: 2.829 / stop criter: 6.144 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 172 / bw: 2.956 / grad: -13.166 / stop criter: 2.829 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 172 / bw: 2.956 / grad: -13.166 / stop criter: 2.829 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 173 / bw: 2.965 / grad: -31.567 / stop criter: 13.166 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 173 / bw: 2.965 / grad: -31.567 / stop criter: 13.166 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 174 / bw: 2.987 / grad: -8.668 / stop criter: 31.567 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s] * GD progress: iter: 174 / bw: 2.987 / grad: -8.668 / stop criter: 31.567 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 175 / bw: 2.992 / grad: -5.757 / stop criter: 8.668 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s] * GD progress: iter: 175 / bw: 2.992 / grad: -5.757 / stop criter: 8.668 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 176 / bw: 2.996 / grad: -9.521 / stop criter: 5.757 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 176 / bw: 2.996 / grad: -9.521 / stop criter: 5.757 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 177 / bw: 3.003 / grad: -1.507 / stop criter: 9.521 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 177 / bw: 3.003 / grad: -1.507 / stop criter: 9.521 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 178 / bw: 3.004 / grad: -17.168 / stop criter: 1.507 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 178 / bw: 3.004 / grad: -17.168 / stop criter: 1.507 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 179 / bw: 3.015 / grad: -0.378 / stop criter: 17.168 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 179 / bw: 3.015 / grad: -0.378 / stop criter: 17.168 :  56%|█████▌    | 168/300 [00:01<00:01, 104.53it/s]* GD progress: iter: 179 / bw: 3.015 / grad: -0.378 / stop criter: 17.168 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 180 / bw: 3.016 / grad: 21.006 / stop criter: 0.378 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s] * GD progress: iter: 180 / bw: 3.016 / grad: 21.006 / stop criter: 0.378 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 181 / bw: 3.001 / grad: -14.109 / stop criter: 21.006 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 181 / bw: 3.001 / grad: -14.109 / stop criter: 21.006 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 182 / bw: 3.011 / grad: 51.168 / stop criter: 14.109 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s] * GD progress: iter: 182 / bw: 3.011 / grad: 51.168 / stop criter: 14.109 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 183 / bw: 2.977 / grad: 18.023 / stop criter: 51.168 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 183 / bw: 2.977 / grad: 18.023 / stop criter: 51.168 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 184 / bw: 2.965 / grad: 25.204 / stop criter: 18.023 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 184 / bw: 2.965 / grad: 25.204 / stop criter: 18.023 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 185 / bw: 2.948 / grad: 20.153 / stop criter: 25.204 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 185 / bw: 2.948 / grad: 20.153 / stop criter: 25.204 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 186 / bw: 2.935 / grad: -2.164 / stop criter: 20.153 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 186 / bw: 2.935 / grad: -2.164 / stop criter: 20.153 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 187 / bw: 2.937 / grad: -40.907 / stop criter: 2.164 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 187 / bw: 2.937 / grad: -40.907 / stop criter: 2.164 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 188 / bw: 2.963 / grad: 25.319 / stop criter: 40.907 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 188 / bw: 2.963 / grad: 25.319 / stop criter: 40.907 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 189 / bw: 2.947 / grad: 1.081 / stop criter: 25.319 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s] * GD progress: iter: 189 / bw: 2.947 / grad: 1.081 / stop criter: 25.319 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 190 / bw: 2.946 / grad: -13.199 / stop criter: 1.081 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 190 / bw: 2.946 / grad: -13.199 / stop criter: 1.081 :  60%|█████▉    | 179/300 [00:01<00:01, 101.26it/s]* GD progress: iter: 190 / bw: 2.946 / grad: -13.199 / stop criter: 1.081 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 191 / bw: 2.955 / grad: -5.542 / stop criter: 13.199 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 191 / bw: 2.955 / grad: -5.542 / stop criter: 13.199 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 192 / bw: 2.958 / grad: -39.654 / stop criter: 5.542 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 192 / bw: 2.958 / grad: -39.654 / stop criter: 5.542 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 193 / bw: 2.983 / grad: -3.365 / stop criter: 39.654 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 193 / bw: 2.983 / grad: -3.365 / stop criter: 39.654 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 194 / bw: 2.986 / grad: -8.651 / stop criter: 3.365 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s] * GD progress: iter: 194 / bw: 2.986 / grad: -8.651 / stop criter: 3.365 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 195 / bw: 2.991 / grad: 3.881 / stop criter: 8.651 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s] * GD progress: iter: 195 / bw: 2.991 / grad: 3.881 / stop criter: 8.651 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 196 / bw: 2.989 / grad: -1.296 / stop criter: 3.881 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 196 / bw: 2.989 / grad: -1.296 / stop criter: 3.881 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 197 / bw: 2.989 / grad: -33.287 / stop criter: 1.296 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 197 / bw: 2.989 / grad: -33.287 / stop criter: 1.296 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 198 / bw: 3.010 / grad: -45.906 / stop criter: 33.287 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 198 / bw: 3.010 / grad: -45.906 / stop criter: 33.287 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 199 / bw: 3.039 / grad: 54.619 / stop criter: 45.906 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s] * GD progress: iter: 199 / bw: 3.039 / grad: 54.619 / stop criter: 45.906 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 200 / bw: 3.005 / grad: -0.089 / stop criter: 54.619 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 200 / bw: 3.005 / grad: -0.089 / stop criter: 54.619 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 201 / bw: 3.005 / grad: -8.624 / stop criter: 0.089 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s] * GD progress: iter: 201 / bw: 3.005 / grad: -8.624 / stop criter: 0.089 :  63%|██████▎   | 190/300 [00:01<00:01, 100.61it/s]* GD progress: iter: 201 / bw: 3.005 / grad: -8.624 / stop criter: 0.089 :  67%|██████▋   | 201/300 [00:01<00:00, 101.36it/s]* GD progress: iter: 202 / bw: 3.010 / grad: -6.350 / stop criter: 8.624 :  67%|██████▋   | 201/300 [00:01<00:00, 101.36it/s]* GD progress: iter: 202 / bw: 3.010 / grad: -6.350 / stop criter: 8.624 :  67%|██████▋   | 201/300 [00:01<00:00, 101.36it/s]* GD progress: iter: 203 / bw: 3.014 / grad: 26.278 / stop criter: 6.350 :  67%|██████▋   | 201/300 [00:01<00:00, 101.36it/s]* GD progress: iter: 203 / bw: 3.014 / grad: 26.278 / stop criter: 6.350 :  67%|██████▋   | 201/300 [00:01<00:00, 101.36it/s]* GD progress: iter: 204 / bw: 2.998 / grad: 24.569 / stop criter: 26.278 :  67%|██████▋   | 201/300 [00:01<00:00, 101.36it/s]* GD progress: iter: 204 / bw: 2.998 / grad: 24.569 / stop criter: 26.278 :  67%|██████▋   | 201/300 [00:01<00:00, 101.36it/s]* GD progress: iter: 205 / bw: 2.983 / grad: 25.277 / stop criter: 24.569 :  67%|██████▋   | 201/300 [00:01<00:00, 101.36it/s]* GD progress: iter: 205 / bw: 2.983 / grad: 25.277 / stop criter: 24.569 :  67%|██████▋   | 201/300 [00:01<00:00, 101.36it/s]* GD progress: iter: 206 / bw: 2.968 / grad: 20.817 / stop criter: 25.277 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 206 / bw: 2.968 / grad: 20.817 / stop criter: 25.277 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 207 / bw: 2.955 / grad: 24.554 / stop criter: 20.817 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 207 / bw: 2.955 / grad: 24.554 / stop criter: 20.817 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 208 / bw: 2.940 / grad: 19.109 / stop criter: 24.554 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 208 / bw: 2.940 / grad: 19.109 / stop criter: 24.554 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 209 / bw: 2.929 / grad: -15.845 / stop criter: 19.109 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 209 / bw: 2.929 / grad: -15.845 / stop criter: 19.109 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 210 / bw: 2.938 / grad: -5.915 / stop criter: 15.845 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s] * GD progress: iter: 210 / bw: 2.938 / grad: -5.915 / stop criter: 15.845 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 211 / bw: 2.942 / grad: -2.419 / stop criter: 5.915 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s] * GD progress: iter: 211 / bw: 2.942 / grad: -2.419 / stop criter: 5.915 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 212 / bw: 2.943 / grad: -10.641 / stop criter: 2.419 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 212 / bw: 2.943 / grad: -10.641 / stop criter: 2.419 :  67%|██████▋   | 201/300 [00:02<00:00, 101.36it/s]* GD progress: iter: 212 / bw: 2.943 / grad: -10.641 / stop criter: 2.419 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 213 / bw: 2.950 / grad: 48.857 / stop criter: 10.641 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 213 / bw: 2.950 / grad: 48.857 / stop criter: 10.641 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 214 / bw: 2.920 / grad: -5.771 / stop criter: 48.857 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 214 / bw: 2.920 / grad: -5.771 / stop criter: 48.857 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 215 / bw: 2.924 / grad: 20.448 / stop criter: 5.771 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s] * GD progress: iter: 215 / bw: 2.924 / grad: 20.448 / stop criter: 5.771 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 216 / bw: 2.912 / grad: -7.204 / stop criter: 20.448 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 216 / bw: 2.912 / grad: -7.204 / stop criter: 20.448 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 217 / bw: 2.916 / grad: -2.864 / stop criter: 7.204 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s] * GD progress: iter: 217 / bw: 2.916 / grad: -2.864 / stop criter: 7.204 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 218 / bw: 2.918 / grad: 28.412 / stop criter: 2.864 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 218 / bw: 2.918 / grad: 28.412 / stop criter: 2.864 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 219 / bw: 2.901 / grad: -7.242 / stop criter: 28.412 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 219 / bw: 2.901 / grad: -7.242 / stop criter: 28.412 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 220 / bw: 2.905 / grad: -3.832 / stop criter: 7.242 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s] * GD progress: iter: 220 / bw: 2.905 / grad: -3.832 / stop criter: 7.242 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 221 / bw: 2.907 / grad: 0.274 / stop criter: 3.832 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s] * GD progress: iter: 221 / bw: 2.907 / grad: 0.274 / stop criter: 3.832 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 222 / bw: 2.907 / grad: 21.556 / stop criter: 0.274 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 222 / bw: 2.907 / grad: 21.556 / stop criter: 0.274 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 223 / bw: 2.895 / grad: -3.513 / stop criter: 21.556 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 223 / bw: 2.895 / grad: -3.513 / stop criter: 21.556 :  71%|███████   | 212/300 [00:02<00:00, 101.29it/s]* GD progress: iter: 223 / bw: 2.895 / grad: -3.513 / stop criter: 21.556 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 224 / bw: 2.897 / grad: -12.907 / stop criter: 3.513 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 224 / bw: 2.897 / grad: -12.907 / stop criter: 3.513 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 225 / bw: 2.904 / grad: 24.020 / stop criter: 12.907 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 225 / bw: 2.904 / grad: 24.020 / stop criter: 12.907 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 226 / bw: 2.891 / grad: 1.648 / stop criter: 24.020 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s] * GD progress: iter: 226 / bw: 2.891 / grad: 1.648 / stop criter: 24.020 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 227 / bw: 2.890 / grad: -8.996 / stop criter: 1.648 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 227 / bw: 2.890 / grad: -8.996 / stop criter: 1.648 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 228 / bw: 2.895 / grad: 21.248 / stop criter: 8.996 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 228 / bw: 2.895 / grad: 21.248 / stop criter: 8.996 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 229 / bw: 2.884 / grad: -15.855 / stop criter: 21.248 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 229 / bw: 2.884 / grad: -15.855 / stop criter: 21.248 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 230 / bw: 2.892 / grad: -6.036 / stop criter: 15.855 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s] * GD progress: iter: 230 / bw: 2.892 / grad: -6.036 / stop criter: 15.855 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 231 / bw: 2.895 / grad: 16.630 / stop criter: 6.036 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s] * GD progress: iter: 231 / bw: 2.895 / grad: 16.630 / stop criter: 6.036 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 232 / bw: 2.886 / grad: 22.213 / stop criter: 16.630 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 232 / bw: 2.886 / grad: 22.213 / stop criter: 16.630 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 233 / bw: 2.875 / grad: -10.361 / stop criter: 22.213 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 233 / bw: 2.875 / grad: -10.361 / stop criter: 22.213 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 234 / bw: 2.880 / grad: -7.821 / stop criter: 10.361 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s] * GD progress: iter: 234 / bw: 2.880 / grad: -7.821 / stop criter: 10.361 :  74%|███████▍  | 223/300 [00:02<00:00, 101.72it/s]* GD progress: iter: 234 / bw: 2.880 / grad: -7.821 / stop criter: 10.361 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s] * GD progress: iter: 235 / bw: 2.884 / grad: 17.664 / stop criter: 7.821 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s] * GD progress: iter: 235 / bw: 2.884 / grad: 17.664 / stop criter: 7.821 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 236 / bw: 2.875 / grad: -5.709 / stop criter: 17.664 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 236 / bw: 2.875 / grad: -5.709 / stop criter: 17.664 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 237 / bw: 2.878 / grad: 79.396 / stop criter: 5.709 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s] * GD progress: iter: 237 / bw: 2.878 / grad: 79.396 / stop criter: 5.709 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 238 / bw: 2.837 / grad: -1.211 / stop criter: 79.396 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 238 / bw: 2.837 / grad: -1.211 / stop criter: 79.396 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 239 / bw: 2.838 / grad: -5.508 / stop criter: 1.211 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s] * GD progress: iter: 239 / bw: 2.838 / grad: -5.508 / stop criter: 1.211 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 240 / bw: 2.841 / grad: -5.462 / stop criter: 5.508 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 240 / bw: 2.841 / grad: -5.462 / stop criter: 5.508 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 241 / bw: 2.843 / grad: 31.827 / stop criter: 5.462 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 241 / bw: 2.843 / grad: 31.827 / stop criter: 5.462 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 242 / bw: 2.828 / grad: -9.849 / stop criter: 31.827 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 242 / bw: 2.828 / grad: -9.849 / stop criter: 31.827 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 243 / bw: 2.832 / grad: -4.160 / stop criter: 9.849 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s] * GD progress: iter: 243 / bw: 2.832 / grad: -4.160 / stop criter: 9.849 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 244 / bw: 2.834 / grad: -21.822 / stop criter: 4.160 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 244 / bw: 2.834 / grad: -21.822 / stop criter: 4.160 :  78%|███████▊  | 234/300 [00:02<00:00, 99.55it/s]* GD progress: iter: 244 / bw: 2.834 / grad: -21.822 / stop criter: 4.160 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 245 / bw: 2.845 / grad: -5.941 / stop criter: 21.822 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 245 / bw: 2.845 / grad: -5.941 / stop criter: 21.822 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 246 / bw: 2.848 / grad: 5.675 / stop criter: 5.941 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]  * GD progress: iter: 246 / bw: 2.848 / grad: 5.675 / stop criter: 5.941 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 247 / bw: 2.845 / grad: -8.252 / stop criter: 5.675 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 247 / bw: 2.845 / grad: -8.252 / stop criter: 5.675 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 248 / bw: 2.849 / grad: -8.240 / stop criter: 8.252 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 248 / bw: 2.849 / grad: -8.240 / stop criter: 8.252 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 249 / bw: 2.853 / grad: -7.004 / stop criter: 8.240 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 249 / bw: 2.853 / grad: -7.004 / stop criter: 8.240 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 250 / bw: 2.857 / grad: -1.720 / stop criter: 7.004 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 250 / bw: 2.857 / grad: -1.720 / stop criter: 7.004 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 251 / bw: 2.857 / grad: 21.885 / stop criter: 1.720 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 251 / bw: 2.857 / grad: 21.885 / stop criter: 1.720 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 252 / bw: 2.847 / grad: 25.218 / stop criter: 21.885 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 252 / bw: 2.847 / grad: 25.218 / stop criter: 21.885 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 253 / bw: 2.835 / grad: 26.650 / stop criter: 25.218 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 253 / bw: 2.835 / grad: 26.650 / stop criter: 25.218 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 254 / bw: 2.822 / grad: -0.594 / stop criter: 26.650 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 254 / bw: 2.822 / grad: -0.594 / stop criter: 26.650 :  81%|████████▏ | 244/300 [00:02<00:00, 99.48it/s]* GD progress: iter: 254 / bw: 2.822 / grad: -0.594 / stop criter: 26.650 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 255 / bw: 2.823 / grad: -3.485 / stop criter: 0.594 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s] * GD progress: iter: 255 / bw: 2.823 / grad: -3.485 / stop criter: 0.594 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 256 / bw: 2.824 / grad: -29.493 / stop criter: 3.485 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 256 / bw: 2.824 / grad: -29.493 / stop criter: 3.485 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 257 / bw: 2.838 / grad: -1.870 / stop criter: 29.493 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 257 / bw: 2.838 / grad: -1.870 / stop criter: 29.493 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 258 / bw: 2.839 / grad: -9.513 / stop criter: 1.870 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s] * GD progress: iter: 258 / bw: 2.839 / grad: -9.513 / stop criter: 1.870 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 259 / bw: 2.843 / grad: 49.963 / stop criter: 9.513 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 259 / bw: 2.843 / grad: 49.963 / stop criter: 9.513 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 260 / bw: 2.820 / grad: -33.763 / stop criter: 49.963 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 260 / bw: 2.820 / grad: -33.763 / stop criter: 49.963 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 261 / bw: 2.836 / grad: -3.710 / stop criter: 33.763 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s] * GD progress: iter: 261 / bw: 2.836 / grad: -3.710 / stop criter: 33.763 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 262 / bw: 2.837 / grad: -0.937 / stop criter: 3.710 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s] * GD progress: iter: 262 / bw: 2.837 / grad: -0.937 / stop criter: 3.710 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 263 / bw: 2.838 / grad: -8.156 / stop criter: 0.937 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 263 / bw: 2.838 / grad: -8.156 / stop criter: 0.937 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 264 / bw: 2.842 / grad: -8.937 / stop criter: 8.156 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 264 / bw: 2.842 / grad: -8.937 / stop criter: 8.156 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 265 / bw: 2.846 / grad: -63.562 / stop criter: 8.937 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 265 / bw: 2.846 / grad: -63.562 / stop criter: 8.937 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 266 / bw: 2.875 / grad: -19.040 / stop criter: 63.562 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 266 / bw: 2.875 / grad: -19.040 / stop criter: 63.562 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 267 / bw: 2.884 / grad: -4.291 / stop criter: 19.040 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s] * GD progress: iter: 267 / bw: 2.884 / grad: -4.291 / stop criter: 19.040 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 268 / bw: 2.886 / grad: 19.363 / stop criter: 4.291 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s] * GD progress: iter: 268 / bw: 2.886 / grad: 19.363 / stop criter: 4.291 :  85%|████████▍ | 254/300 [00:02<00:00, 99.60it/s]* GD progress: iter: 268 / bw: 2.886 / grad: 19.363 / stop criter: 4.291 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 269 / bw: 2.877 / grad: -3.558 / stop criter: 19.363 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 269 / bw: 2.877 / grad: -3.558 / stop criter: 19.363 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 270 / bw: 2.879 / grad: -11.110 / stop criter: 3.558 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 270 / bw: 2.879 / grad: -11.110 / stop criter: 3.558 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 271 / bw: 2.884 / grad: -8.926 / stop criter: 11.110 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 271 / bw: 2.884 / grad: -8.926 / stop criter: 11.110 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 272 / bw: 2.888 / grad: -4.872 / stop criter: 8.926 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s] * GD progress: iter: 272 / bw: 2.888 / grad: -4.872 / stop criter: 8.926 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 273 / bw: 2.890 / grad: -2.997 / stop criter: 4.872 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 273 / bw: 2.890 / grad: -2.997 / stop criter: 4.872 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 274 / bw: 2.891 / grad: -3.376 / stop criter: 2.997 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 274 / bw: 2.891 / grad: -3.376 / stop criter: 2.997 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 275 / bw: 2.893 / grad: -5.353 / stop criter: 3.376 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 275 / bw: 2.893 / grad: -5.353 / stop criter: 3.376 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 276 / bw: 2.895 / grad: 56.448 / stop criter: 5.353 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 276 / bw: 2.895 / grad: 56.448 / stop criter: 5.353 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 277 / bw: 2.870 / grad: 23.717 / stop criter: 56.448 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 277 / bw: 2.870 / grad: 23.717 / stop criter: 56.448 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 278 / bw: 2.859 / grad: -5.234 / stop criter: 23.717 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 278 / bw: 2.859 / grad: -5.234 / stop criter: 23.717 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 279 / bw: 2.861 / grad: -13.604 / stop criter: 5.234 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 279 / bw: 2.861 / grad: -13.604 / stop criter: 5.234 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 280 / bw: 2.867 / grad: -2.976 / stop criter: 13.604 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 280 / bw: 2.867 / grad: -2.976 / stop criter: 13.604 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 281 / bw: 2.869 / grad: -1.777 / stop criter: 2.976 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s] * GD progress: iter: 281 / bw: 2.869 / grad: -1.777 / stop criter: 2.976 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 282 / bw: 2.869 / grad: 23.642 / stop criter: 1.777 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 282 / bw: 2.869 / grad: 23.642 / stop criter: 1.777 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 283 / bw: 2.859 / grad: 50.537 / stop criter: 23.642 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 283 / bw: 2.859 / grad: 50.537 / stop criter: 23.642 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 284 / bw: 2.837 / grad: 8.132 / stop criter: 50.537 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s] * GD progress: iter: 284 / bw: 2.837 / grad: 8.132 / stop criter: 50.537 :  89%|████████▉ | 268/300 [00:02<00:00, 110.99it/s]* GD progress: iter: 284 / bw: 2.837 / grad: 8.132 / stop criter: 50.537 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 285 / bw: 2.833 / grad: -8.142 / stop criter: 8.132 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 285 / bw: 2.833 / grad: -8.142 / stop criter: 8.132 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 286 / bw: 2.837 / grad: 16.118 / stop criter: 8.142 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 286 / bw: 2.837 / grad: 16.118 / stop criter: 8.142 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 287 / bw: 2.830 / grad: -9.630 / stop criter: 16.118 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 287 / bw: 2.830 / grad: -9.630 / stop criter: 16.118 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 288 / bw: 2.834 / grad: -33.427 / stop criter: 9.630 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 288 / bw: 2.834 / grad: -33.427 / stop criter: 9.630 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 289 / bw: 2.848 / grad: 18.917 / stop criter: 33.427 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 289 / bw: 2.848 / grad: 18.917 / stop criter: 33.427 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 290 / bw: 2.840 / grad: -6.932 / stop criter: 18.917 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 290 / bw: 2.840 / grad: -6.932 / stop criter: 18.917 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 291 / bw: 2.843 / grad: -32.365 / stop criter: 6.932 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 291 / bw: 2.843 / grad: -32.365 / stop criter: 6.932 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 292 / bw: 2.856 / grad: -7.011 / stop criter: 32.365 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 292 / bw: 2.856 / grad: -7.011 / stop criter: 32.365 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 293 / bw: 2.859 / grad: 1.471 / stop criter: 7.011 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]  * GD progress: iter: 293 / bw: 2.859 / grad: 1.471 / stop criter: 7.011 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 294 / bw: 2.859 / grad: -34.528 / stop criter: 1.471 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 294 / bw: 2.859 / grad: -34.528 / stop criter: 1.471 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 295 / bw: 2.873 / grad: -7.385 / stop criter: 34.528 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 295 / bw: 2.873 / grad: -7.385 / stop criter: 34.528 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 296 / bw: 2.876 / grad: 19.798 / stop criter: 7.385 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s] * GD progress: iter: 296 / bw: 2.876 / grad: 19.798 / stop criter: 7.385 :  95%|█████████▍| 284/300 [00:02<00:00, 119.42it/s]* GD progress: iter: 296 / bw: 2.876 / grad: 19.798 / stop criter: 7.385 :  99%|█████████▊| 296/300 [00:02<00:00, 114.50it/s]* GD progress: iter: 297 / bw: 2.868 / grad: -4.752 / stop criter: 19.798 :  99%|█████████▊| 296/300 [00:02<00:00, 114.50it/s]* GD progress: iter: 297 / bw: 2.868 / grad: -4.752 / stop criter: 19.798 :  99%|█████████▊| 296/300 [00:02<00:00, 114.50it/s]* GD progress: iter: 298 / bw: 2.870 / grad: -2.388 / stop criter: 4.752 :  99%|█████████▊| 296/300 [00:02<00:00, 114.50it/s] * GD progress: iter: 298 / bw: 2.870 / grad: -2.388 / stop criter: 4.752 :  99%|█████████▊| 296/300 [00:02<00:00, 114.50it/s]* GD progress: iter: 299 / bw: 2.871 / grad: -4.667 / stop criter: 2.388 :  99%|█████████▊| 296/300 [00:02<00:00, 114.50it/s]* GD progress: iter: 299 / bw: 2.871 / grad: -4.667 / stop criter: 2.388 :  99%|█████████▊| 296/300 [00:02<00:00, 114.50it/s]* GD progress: iter: 300 / bw: 2.873 / grad: -5.809 / stop criter: 4.667 :  99%|█████████▊| 296/300 [00:02<00:00, 114.50it/s]* GD progress: iter: 300 / bw: 2.873 / grad: -5.809 / stop criter: 4.667 :  99%|█████████▊| 296/300 [00:02<00:00, 114.50it/s]* GD progress: iter: 300 / bw: 2.873 / grad: -5.809 / stop criter: 4.667 : 100%|██████████| 300/300 [00:02<00:00, 105.27it/s]

Now, let’s compare it to the previous example.

print("Estimated bandwidth :" + str(gc2_fit.optimization_outputs['opt_bandwidth']))
gc2_fit.draw_learning_curve()

Estimated bandwidth :2.8727792285382594

Compare MSE and MAPE.

y_pred2 = gc2_fit.predict(X_test1)
print(mean_absolute_percentage_error(y_test1, y_pred2))
print(mean_squared_error(y_test1, y_pred2))

0.2185872702257307
162.5682850501545

Compare qq-plot.

gc2_fit.draw_learning_curve(y_test=y_test1, fig_type='qq')

`GradientCOBRA` on real dataset

We look at the California housing dataset from sklearn.datasets module. To illustrate the idea, we only work with the first \(1000\) observations.

from sklearn.datasets import fetch_california_housing
data = fetch_california_housing()
X_real, y_real = data['data'], data['target']

X_train_real, X_test_real, y_train_real, y_test_real = train_test_split(X_real[:1000,:], y_real[:1000], test_size=0.2)
print('shape: x_train = {} , x_train = {} , y_train = {} , y_test = {}'.format(X_train_real.shape, X_test_real.shape, y_train_real.shape, y_test_real.shape))

shape: x_train = (800, 8) , x_train = (200, 8) , y_train = (800,) , y_test = (200,)

We gave some random parameters to the method as follows.

gc_real = GradientCOBRA(opt_method="grad",
                        learning_rate=0.01,
                        estimator_list=['random_forest', 'knn', 'ridge', 'lasso'],
                        estimator_params={
                                'random_forest' : {'n_estimators': 300},
                                'knn' : {'n_neighbors' : 10}
                        })
gc_real_fit = gc_real.fit(X_train_real, y_train_real)

* GD progress: iter: 1 / bw: 3.000 / grad: 0.016 / stop criter: 1.000 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 1 / bw: 2.990 / grad: 0.016 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 1 / bw: 2.990 / grad: 0.016 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 2 / bw: 2.990 / grad: 0.016 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 2 / bw: 2.990 / grad: 0.016 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 3 / bw: 2.990 / grad: 0.016 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 3 / bw: 2.990 / grad: 0.016 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 4 / bw: 2.990 / grad: 0.016 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 4 / bw: 2.990 / grad: 0.016 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 5 / bw: 2.990 / grad: 0.011 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 5 / bw: 2.990 / grad: 0.011 / stop criter: 0.016 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 6 / bw: 2.984 / grad: 0.002 / stop criter: 0.011 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 6 / bw: 2.984 / grad: 0.002 / stop criter: 0.011 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 7 / bw: 2.982 / grad: 0.027 / stop criter: 0.002 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 7 / bw: 2.982 / grad: 0.027 / stop criter: 0.002 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 7 / bw: 2.982 / grad: 0.027 / stop criter: 0.002 :   2%|▏         | 7/300 [00:00<00:03, 76.99it/s]

Now, let’s look at the obtained bandwidth and the optimization result.

print("Optimal bandwidth: {}".format(gc_real_fit.optimization_outputs['opt_bandwidth']))
gc_real_fit.draw_learning_curve()

Optimal bandwidth: 2.965913414265725

We look at the numerical and graphical performance.

y_pred_real = gc_real_fit.predict(X_test_real)
print(mean_absolute_percentage_error(y_test_real, y_pred_real))
print(mean_squared_error(y_test_real, y_pred_real))
gc_real_fit.draw_learning_curve(y_test=y_test_real, fig_type='qq')

0.1788263339928227
0.2356169878883068

A comparison with `Adaboost`

We campare the fitted method on California data with Adaboost method.

from sklearn.ensemble import AdaBoostRegressor

ada = AdaBoostRegressor(n_estimators=1000)
ada_fit = ada.fit(X_train_real, y_train_real)
ada_pred = ada_fit.predict(X_test_real)
print(mean_absolute_percentage_error(y_test_real, ada_pred))
print(mean_squared_error(y_test_real, ada_pred))

0.28230461901894716
0.31522759465126454

Pretrained basic estimators

An interesting application of consensual aggregation methods is having pretrained estimators from some source, then applying them to other testing data (not necessarily from the same source, having the same inputs is enough). The only requirement is the predictability of the basic estimators on new the observations, and only the predicted features are used in the aggregation. Here, we build pretrained estimators including XGBoost then aggregate it to some sklearn basic estimators.

We first split the training data into two parts: \(X_k\) for building basic estimators, and \(X_\ell\) for aggregation. We use the constructed estimators to predict the test data and only those predictions are used in the final predictions.

import numpy as np
from sklearn.ensemble import RandomForestRegressor
from sklearn.linear_model import LinearRegression
from sklearn.neighbors import KNeighborsRegressor
from sklearn.tree import DecisionTreeRegressor

# X_k and X_l split
id_k = np.random.permutation(range(len(y_train_real)))
k =int(.5 * len(y_train_real))
X_k, X_l, y_k, y_l = X_train_real[id_k[:k],:], X_train_real[id_k[k:],:], y_train_real[id_k[:k]], y_train_real[id_k[k:]]

# Building basic estiators
rf_real = RandomForestRegressor(n_estimators=300).fit(X_k, y_k)
lm_real = LinearRegression().fit(X_k, y_k)
knn_real = KNeighborsRegressor(n_neighbors=10).fit(X_k, y_k)
tr_real = DecisionTreeRegressor(min_samples_leaf=5).fit(X_k, y_k)

# External XGBoost estiator
import xgboost 
xgb = xgboost.XGBRegressor(n_estimators = 500)
xgb_real = xgb.fit(X_k, y_k)

# All pretrained estimators
basic_estimators = (rf_real, lm_real, knn_real, tr_real, xgb_real)

# Predicted features on X_l for aggregation
pred_feature_l = np.column_stack([est.predict(X_l) for est in basic_estimators])

# Predicted features on Testing data
pred_feature_test = np.column_stack([est.predict(X_test_real) for est in basic_estimators])

To fit the aggregation method on predicted features (NOT the input data), we have to set argument as_predictions = True. This tells the fit method not to build any basic estimators on the given input (which is already the predictions), and the optimization method is performed directly on \(X\).

gc3_fit = GradientCOBRA(learning_rate=0.01, opt_method="grad").fit(X=pred_feature_l,
                              y=y_l,
                              as_predictions=True)

* GD progress: iter: 1 / bw: 3.000 / grad: 0.019 / stop criter: 1.000 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 1 / bw: 2.990 / grad: 0.019 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 1 / bw: 2.990 / grad: 0.019 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 2 / bw: 2.990 / grad: 0.019 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 2 / bw: 2.990 / grad: 0.019 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 3 / bw: 2.990 / grad: 0.019 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 3 / bw: 2.990 / grad: 0.019 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 4 / bw: 2.990 / grad: 0.019 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 4 / bw: 2.990 / grad: 0.019 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 5 / bw: 2.990 / grad: 0.021 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 5 / bw: 2.990 / grad: 0.021 / stop criter: 0.019 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 6 / bw: 2.979 / grad: -0.010 / stop criter: 0.021 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 6 / bw: 2.979 / grad: -0.010 / stop criter: 0.021 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 7 / bw: 2.984 / grad: 0.002 / stop criter: 0.010 :   0%|          | 0/300 [00:00<?, ?it/s] * GD progress: iter: 7 / bw: 2.984 / grad: 0.002 / stop criter: 0.010 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 8 / bw: 2.983 / grad: 0.041 / stop criter: 0.002 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 8 / bw: 2.983 / grad: 0.041 / stop criter: 0.002 :   0%|          | 0/300 [00:00<?, ?it/s]* GD progress: iter: 8 / bw: 2.983 / grad: 0.041 / stop criter: 0.002 :   3%|▎         | 8/300 [00:00<00:03, 86.43it/s]

We look at the optimization algorithm performance.

print("Estimated bandwidth :" + str(gc3_fit.optimization_outputs['opt_bandwidth']))
gc3_fit.draw_learning_curve()

Estimated bandwidth :2.962236440290544

Now, let’s look at the result.

pred_add = gc3_fit.predict(pred_feature_test)
print(mean_absolute_percentage_error(y_test_real, pred_add))
print(mean_squared_error(y_test_real, pred_add))

0.1527194278509763
0.1909836867420075

GradientCobra Python Library

Introduction to GradientCOBRA method

GradientCOBRA method

Parameters

Returns:

Methods:

Installation of the library from `pypi`

Implementation of the method

Importing some packages

`GradientCOBRA` on simulated data

\(\bullet\) `GradientCOBRA` with default parameters

\(\bullet\) `GradientCOBRA` with non-default parameters

`GradientCOBRA` on real dataset

A comparison with `Adaboost`

Pretrained basic estimators

References

Read also:

Introduction to GradientCOBRA method

GradientCOBRA method

Parameters

Returns:

Methods:

Installation of the library from pypi

Implementation of the method

Importing some packages

GradientCOBRA on simulated data

\(\bullet\) GradientCOBRA with default parameters

\(\bullet\) GradientCOBRA with non-default parameters

GradientCOBRA on real dataset

A comparison with Adaboost

Pretrained basic estimators

References

Read also:

Installation of the library from `pypi`

`GradientCOBRA` on simulated data

\(\bullet\) `GradientCOBRA` with default parameters

\(\bullet\) `GradientCOBRA` with non-default parameters

`GradientCOBRA` on real dataset

A comparison with `Adaboost`