API

hyperts.experiment

hyperts.experiment.make_experiment(train_data, task, eval_data=None, test_data=None, mode='stats', max_trials=50, eval_size=0.2, cv=False, num_folds=3, ensemble_size=10, target=None, freq=None, timestamp=None, forecast_train_data_periods=None, forecast_drop_part_sample=False, timestamp_format='%Y-%m-%d %H:%M:%S', covariates=None, dl_forecast_window=None, dl_forecast_horizon=1, contamination=0.05, id=None, searcher=None, search_space=None, search_callbacks=None, searcher_options=None, callbacks=None, early_stopping_rounds=20, early_stopping_time_limit=3600, early_stopping_reward=None, reward_metric=None, optimize_direction=None, discriminator=None, hyper_model_options=None, tf_gpu_usage_strategy=0, tf_memory_limit=2048, final_retrain_on_wholedata=True, verbose=1, log_level=None, random_state=None, clear_cache=None, **kwargs)

Parameters:

• train_data (str, Pandas or Dask or Cudf DataFrame.) – Feature data for training with target column. For str, it’s should be the data path in file system, will be loaded as pnadas Dataframe. we’ll detect data format from this path (only .csv and .parquet are supported now).

• task (str.) – Task could be ‘univariate-forecast’, ‘multivariate-forecast’, and ‘univariate-binaryclass’, ‘univariate-multiclass’, ‘multivariate-binaryclass, and ’multivariate-multiclass’. Notably, task can also configure ‘forecast’, ‘classification’, ‘regression’，and ‘detection’. Besides, ‘tsf’, ‘utsf’，’mtsf’, ‘tsc’, ‘tsr’, ‘tsd’(‘tsa’, ‘tsad’) are also ok. At this point, HyprTS will perform detailed task type inference from the data combined with other known column information.

• eval_data (str, Pandas or Dask or Cudf DataFrame, optional.) – Feature data for evaluation, should be None or have the same python type with ‘train_data’.

• test_data (str, Pandas or Dask or Cudf DataFrame, optional.) – Feature data for testing without target column, should be None or have the same python type with ‘train_data’.

• max_trials (int, maximum number of tests (model search), optional, (default=50).) –

• eval_size (float or int, When the eval_data is None, customize the ratio to split the eval_data from) – the train_data. int indicates the prediction length of the forecast task. (default=0.2 or 10).

• cv (bool, default False.) – If True, use cross-validation instead of evaluation set reward to guide the search process.

• num_folds (int, default 3.) – Number of cross-validated folds, only valid when cv is true.

• mode (str, default 'stats'. Optional {'stats', 'dl', 'nas'}, where,) – ‘stats’ indicates that all the models selected in the execution experiment are statistical models. ‘dl’ indicates that all the models selected in the execution experiment are deep learning models. ‘nas’ indicates that the selected model of the execution experiment will be a deep network model for neural architecture search, which is not currently supported.

• target (str or list, optional.) – Target feature name for training, which must be one of the train_data columns for classification[str], regression[str] or unvariate forecast task [list]. For multivariate forecast task, it is multiple columns of training data.

• ensemble_size ('int' or None, default 10.) – The number of estimator to ensemble. During the AutoML process, a lot of models will be generated with different preprocessing pipelines, different models, and different hyperparameters. Usually selecting some of the models that perform well to ensemble can obtain better generalization ability than just selecting the single best model.

• freq ('str', DateOffset or None, default None.) – Note: If your task is a discontinuous time series, you can specify the freq as ‘Discrete’.

• timestamp (str, forecast task 'timestamp' cannot be None, (default=None).) –

• forecast_train_data_periods ('int', Cut off a certain period of data from the train data from back to front) – as a train set. (default=None).

• timestamp_format (str, the date format of timestamp col for forecast task, (default='%Y-%m-%d %H:%M:%S').) –

• covariates/covariables (list[n*str], if the data contains covariates, specify the covariable column names,) – (default=None).

• dl_forecast_window (int, list or None. When selecting 'dl' or 'nas' mode, you can specify window, which is the) – sequence length of each sample (lag), (default=None).

• dl_forecast_horizon (int or None. When selecting 'dl' or 'nas' mode, you can specify horizon, which is the length) – of the interval between the input and the target, (default=1).

• contamination (float, should be in the interval (0, 1], optional (default=0.05).) – This parameter is adopted only in anomaly detection task to generate pseudo ground truth. The amount of contamination of the data set, i.e. the proportion of outliers in the data set. Used when fitting to define the threshold on the scores of the samples.

• id (str or None, (default=None).) – The experiment id.

• callbacks (list of ExperimentCallback, optional.) – ExperimentCallback list.

• searcher (str, searcher class, search object, optional.) – The hypernets Searcher instance to explore search space, default is MCTSSearcher instance. For str, should be one of ‘evolution’, ‘mcts’, ‘random’. For class, should be one of EvolutionSearcher, MCTSSearcher, RandomSearcher, or subclass of hypernets Searcher. For other, should be instance of hypernets Searcher.

• searcher_options (dict, optional, default is None.) – The options to create searcher, is used if searcher is str.

• search_space (callable, optional) – Used to initialize searcher instance (if searcher is None, str or class).

• search_callbacks – Hypernets search callbacks, used to initialize searcher instance (if searcher is None, str or class). If log_level >= WARNNING, default is EarlyStoppingCallback only. If log_level < WARNNING, defalult is EarlyStoppingCallback plus SummaryCallback.

• early_stopping_rounds (int optional.) – Setting of EarlyStoppingCallback, is used if EarlyStoppingCallback instance not found from search_callbacks. Set zero or None to disable it, default is 20.

• early_stopping_time_limit (int, optional.) – Setting of EarlyStoppingCallback, is used if EarlyStoppingCallback instance not found from search_callbacks. Set zero or None to disable it, default is 3600 seconds.

• early_stopping_reward (float, optional.) – Setting of EarlyStoppingCallback, is used if EarlyStoppingCallback instance not found from search_callbacks. Set zero or None to disable it, default is None.

• reward_metric (str, callable, optional, (default 'accuracy' for binary/multiclass task, 'rmse' for) –

  forecast/regression task) Hypernets search reward metric name or callable. Possible values:

  • accuracy

  • auc

  • f1

  • logloss

  • mse

  • mae

  • rmse

  • mape

  • smape

  • msle

  • precision

  • r2

  • recall

• optimize_direction (str, optional.) – Hypernets search reward metric direction, default is detected from reward_metric.

• discriminator (instance of hypernets.discriminator.BaseDiscriminator, optional) – Discriminator is used to determine whether to continue training

• hyper_model_options (dict, optional.) – Options to initlize HyperModel except reward_metric, task, callbacks, discriminator.

• tf_gpu_usage_strategy (int, optional {0, 1, 2}.) – Deep neural net models(tensorflow) gpu usage strategy. 0:cpu | 1:gpu-memory growth | 2: gpu-memory limit.

• tf_memory_limit (int, GPU memory limit, default 2048.) –

• final_retrain_on_wholedata (bool, after the search, whether to retrain the optimal model on the whole data set.) – default True.

• random_state (int or None, default None.) –

• clear_cache (bool, optional, (default False)) – Clear cache store before running the expeirment.

• verbose (int, 0, 1, or 2, (default=1).) – 0 = silent, 1 = progress bar, 2 = one line per epoch (DL mode). Print order selection output to the screen.

• log_level (int, str, or None, (default=None),) –

  Level of logging, possible values:

  -logging.CRITICAL -logging.FATAL -logging.ERROR -logging.WARNING -logging.WARN -logging.INFO -logging.DEBUG -logging.NOTSET

• kwargs – Parameters to initialize experiment instance, refrence TSCompeteExperiment for more details.

Return type:

Runnable experiment object.

hyperts.hyper_ts

class hyperts.hyper_ts.HyperTSEstimator(task, mode, reward_metric, space_sample, timestamp=None, covariates=None, data_cleaner_params=None, weights_cache=None)

Bases: Estimator

A Estimator object about Time Series.

Parameters:

• task ('str'.) – Task could be ‘univariate-forecast’, ‘multivariate-binaryclass’, etc. See consts.py for details.

• mode ('str'.) – The hyperts can support three mode: ‘dl’, ‘stats’, and ‘nas’.

• space_sample (An instance class representing a hyperts estimator.) –

• data_cleaner_params ('dirt' or None, default None.) – For details of parameters, refer to hypernets.tabular.data_cleaner.

build_pipeline(space, last_transformer)

summary()

fit_cross_validation(X, y, verbose=0, stratified=True, num_folds=3, pos_label=None, shuffle=False, random_state=9527, metrics=None, **kwargs)

get_scores(y, oof_, metrics)

get_iteration_scores()

fit(X, y, pos_label=None, verbose=0, **kwargs)

predict(X, verbose=0, **kwargs)

predict_proba(X, verbose=0, **kwargs)

fit_transform_X(X)

proba2predict(proba, proba_threshold=0.5)

evaluate(X, y, metrics=None, verbose=0, **kwargs)

save(model_file, external=False)

static load(model_file)

set_discriminator(discriminator)

class hyperts.hyper_ts.HyperTS(searcher, task=None, mode='stats', timestamp=None, covariates=None, dispatcher=None, callbacks=None, reward_metric='accuracy', discriminator=None, data_cleaner_params=None, use_layer_weight_cache=False)

Bases: HyperModel

A HyperModel object about Time Series.

Parameters:

• searcher ('str', searcher class, search object.) – Searchers come from hypernets, such as EvolutionSearcher, MCTSSearcher, or RandomSearcher, etc. See hypernets.searchers for details.

• task ('str' or None, default None.) – Task could be ‘univariate-forecast’, ‘multivariate-forecast’, and ‘univariate-binaryclass’, etc. See consts.py for details.

• mode ('str', default 'stats'.) – The hyperts can support three mode: ‘dl’, ‘stats’, and ‘nas’.

• dispatcher (class object or None, default None.) – Dispatcher is used to provide different execution modes for search trials, such as in process mode (InProcessDispatcher), distributed parallel mode (DaskDispatcher), etc.

• callbacks (list of ExperimentCallback or None, default None.) –

• reward_metric ('str' or callable.) – Default ‘accuracy’ for binary/multiclass task, ‘rmse’ for forecast/regression task.

• discriminator (Instance of hypernets.discriminator.BaseDiscriminator, which is used to determine) – whether to continue training. Default None.

• data_cleaner_params ('dirt' or None, default None.) – For details of parameters, refer to hypernets.tabular.data_cleaner.

• clear_cache ('bool', default False.) – Clear cache store before running the expeirment.

Returns:

hyper_ts_cls – Subclass of HyperModel to run trials within the experiment.

Return type:

subclass of HyperModel

load_estimator(model_file)

export_trial_configuration(trial)

property best_reward

property best_trial_no

export_configuration(trials)

final_train(space_sample, X, y, **kwargs)

generate_dataset_id(X, y)

get_best_trial()

get_top_trials(top_n)

infer_task_type(y)

plot_hyperparams(destination='notebook', output='hyperparams.html')

search(X, y, X_eval, y_eval, cv=False, num_folds=3, max_trials=3, dataset_id=None, trial_store=None, **fit_kwargs)

Parameters:

• X – Pandas or Dask DataFrame, feature data for training

• y – Pandas or Dask Series, target values for training

• X_eval – (Pandas or Dask DataFrame) or None, feature data for evaluation

• y_eval – (Pandas or Dask Series) or None, target values for evaluation

• cv – Optional, int(default=False), If set to true, use cross-validation instead of evaluation set reward to guide the search process

• num_folds – Optional, int(default=3), Number of cross-validated folds, only valid when cv is true

• max_trials – Optional, int(default=10), The upper limit of the number of search trials, the search process stops when the number is exceeded

• dataset_id –

• trial_store –

• fit_kwargs – Optional, dict, parameters for fit method of model

Returns:

hyperts.toolbox

hyperts.toolbox.from_3d_array_to_nested_df(data: ndarray, columns: Optional[str] = None, cells_as_array: bool = False)

Convert Numpy ndarray with shape (nb_samples, series_length, nb_variables) into nested pandas DataFrame (with time series as numpy array or pandas Series in cells)

Parameters:

• data (np.ndarray) – 3-dimensional Numpy array to convert to nested pandas DataFrame format

• columns (list-like, default = None) – Optional list of names to use for naming nested DataFrame’s columns

• cells_as_array (bool, default = False) – If True, then nested cells contain Numpy array If False, then nested cells contain pandas Series

Returns:

df

Return type:

pd.DataFrame

hyperts.toolbox.from_nested_df_to_3d_array(data: DataFrame)

Convert nested pandas DataFrame (with time series as numpy array or pandas Series in cells) into Numpy ndarray with shape (nb_samples, series_length, nb_variables).

Parameters:

data (pd.DataFrame) – Nested pandas DataFrame

Returns:

data_3d – 3-dimensional NumPy array

Return type:

np.arrray

hyperts.toolbox.random_train_test_split(*arrays, test_size=None, train_size=None, random_state=None, shuffle=True, stratify=None)

Split arrays or matrices into random train and test subsets. This is a wrapper of scikit-learn’s train_test_split that has shuffle.

hyperts.toolbox.temporal_train_test_split(*arrays, test_size=None, train_size=None, test_horizon=None)

Split arrays or matrices into sequential train and test subsets.This is a wrapper of scikit-learn’s train_test_split that does not shuffle.

Parameters:

• *arrays (sequence of indexables with same length / shape[0] Allowed inputs) –

• lists (are) –

• arrays (numpy) –

• dataframes. (scipy-sparse matrices or pandas) –

• test_size (float, int or None, optional (default=None)) – If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it will be set to 0.25.

• train_size (float, int, or None, (default=None)) – If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int, represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.

• test_horizon (int or None, (default=None)) – If int, represents the forecast horizon length.

Returns:

splitting – List containing train-test split of inputs.

Return type:

list, length=2 * len(arrays)

hyperts.toolbox.metafeatures_from_timeseries(x: DataFrame, timestamp: str, period=None, scale_ts=True, freq_mapping_dict=None, features_list=None)

Extracting the meta-features of time series.

Parameters:

• x (pd.DataFrame, the time series.) –

• timestamp (str, timestamp name of x.) –

• period (int or None, the seasonal of the time series, default None.) –

• scale_ts (bool, whether scale original time series.) –

• freq_mapping_dict (24, 'D': 7, 'W': 54, 'M': 12,) – ‘Q’: 4, ‘Y’: 1, ‘A’: 1, ‘S’: 60, ‘T’: 60}.

• dict (24, 'D': 7, 'W': 54, 'M': 12,) – ‘Q’: 4, ‘Y’: 1, ‘A’: 1, ‘S’: 60, ‘T’: 60}.

• {'H' (default) – ‘Q’: 4, ‘Y’: 1, ‘A’: 1, ‘S’: 60, ‘T’: 60}.

• features_list –

• List[str] –

• ['simple' (default) –

• 'all']. –

hyperts.toolbox.generate_anomaly_pseudo_ground_truth(X_train, X_test=None, local_region_size: int = 30, local_max_features: float = 1.0, local_region_iterations: int = 20, generate_train_label_type: str = 'iforest', contamination: float = 0.05, random_state=None)

Genrate pseudo ground truth for anomaly detection.

Parameters:

• X_train (numpy array of shape (n_samples, n_features).) –

• X_test (numpy array of shape (n_samples, n_features).) –

• local_region_size (int, optional (default=30)) – Number of training points to consider in each iteration of the local region generation process (30 by default).

• local_max_features (float in (0.5, 1.), optional (default=1.0)) – Maximum proportion of number of features to consider when defining the local region (1.0 by default).

• local_region_iterations (int, optional (default=20)) – Number of iteration of the local region generation process.

• generate_train_label_type (str, optional (default='iforest')) – The method of genetating training pseudo labels.

• contamination ('auto' or float, optional (default=0.05)) – The amount of contamination of the data set, i.e. the proportion of outliers in the data set. Used when fitting to define the threshold on the scores of the samples.

• random_state (RandomState, optional (default=None)) – A random number generator instance to define the state of the random permutations generator.

References

hyperts.utils

utils.metrics

hyperts.utils.metrics.check_is_array(y_true, y_pred)

Check whether the value is array-like. If not, convert the value to array-like.

hyperts.utils.metrics.mse(y_true, y_pred, axis=None)

Mean squared error.

Note that this implementation can handle NaN.

Parameters:

• y_true (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.

• y_pred (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.

Returns:

loss – A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type:

float or ndarray of floats

hyperts.utils.metrics.mae(y_true, y_pred, axis=None)

Mean absolute error.

Note that this implementation can handle NaN.

Parameters:

• y_true (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.

• y_pred (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.

Returns:

loss – A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type:

float or ndarray of floats

hyperts.utils.metrics.rmse(y_true, y_pred)

Root mean squared error.

Note that this implementation can handle NaN.

Parameters:

• y_true (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.

• y_pred (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.

Returns:

loss – A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type:

float or ndarray of floats

hyperts.utils.metrics.mape(y_true, y_pred, epsihon=1e-06, mask=False, axis=None)

Mean absolute percentage error.

Note that this implementation can handle NaN.

Parameters:

• y_true (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.

• y_pred (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.

• epsihon (float, threshold to avoid division by zero. Default is 1e-06.) –

• mask (bool, if True, the mask removes y_ture=0. Default is False.) –

Returns:

loss – A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type:

float or ndarray of floats

hyperts.utils.metrics.smape(y_true, y_pred, axis=None)

Symmetric mean absolute percentage error.

Note that this implementation can handle NaN.

Parameters:

• y_true (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.

• y_pred (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.

Returns:

loss – A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type:

float or ndarray of floats

hyperts.utils.metrics.msle(y_true, y_pred, epsihon=1e-06, axis=None)

Mean squared logarithmic error regression loss.

Note that this implementation can handle NaN and y_pred contains negative values.

Parameters:

• y_true (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.

• y_pred (pd.DataFrame or array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.

Returns:

loss – A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type:

float or ndarray of floats

hyperts.utils.metrics.auc(y_true, y_score, average='macro', sample_weight=None, max_fpr=None, multi_class='raise', labels=None)

Compute Area Under the Receiver Operating Characteristic Curve (ROC AUC) from prediction scores.

Note: this implementation can be used with binary, multiclass and multilabel classification, but some restrictions apply (see sklearn.metrics.roc_auc_score).

hyperts.utils.metrics.calc_score(y_true, y_preds, y_proba=None, metrics=('accuracy',), task='binary', pos_label=None, classes=None, average=None)

hyperts.utils.metrics.metric_to_scorer(metric, task, pos_label=None, **options)

class hyperts.utils.metrics.Metrics

Bases: Metrics

calc_score(y_preds, y_proba=None, metrics=('accuracy',), task='binary', pos_label=None, classes=None, average=None)

metric_to_scorer(task, pos_label=None, **options)

evaluate(X, y, metrics, *, task=None, pos_label=None, classes=None, average=None, threshold=0.5, n_jobs=-1)

metric_to_scoring(task='binary', pos_label=None)

predict(X, *, task=None, classes=None, threshold=0.5, n_jobs=None)

predict_proba(X, *, n_jobs=None)

proba2predict(*, task=None, threshold=0.5, classes=None)

utils.plot

hyperts.utils.plot.plot_plotly(forecast, timestamp_col, target_col, var_id=0, actual=None, history=None, forecast_interval=None, show_forecast_interval=False, include_history=False, **kwargs)

Plots forecast trend curves for the forecst task by plotly.

Notes

1. This function can plot the curve of only one target variable. If not specified, index 0 is ploted by default.

2. This function supports ploting of historical observations, future actual values, and forecast intervals.

Parameters:

• forecast ('DataFrame'. The columns need to include the timestamp column) – and the target columns.

• timestamp_col (str. 'timestamp' column name.) –

• target_col (str or list. target columns name.) –

• var_id ('int' or 'str'. If int, it is the index of the target column. If str,) – it is the name of the target column. default 0.

• actual ('DataFrame' or None. If it is not None, the column needs to include) – the time column and the target column.

• history ('DataFrame'. History data. The columns need to include the timestamp column) – and the target columns.

• forecast_interval ('DataFrame'. Forecast confidence interval.) –

• show_forecast_interval ('bool'. Whether to show the forecast intervals.) – Default False.

• include_history ('bool'. Whether to show the historical observations.) – Default True.

Returns:

fig

Return type:

‘plotly.graph_objects.Figure’.

hyperts.utils.plot.plot_mpl(forecast, timestamp_col, target_col, var_id=0, actual=None, history=None, forecast_interval=None, show_forecast_interval=False, include_history=False, figsize=None, grid=True, **kwargs)

Plots forecast trend curves for the forecst task by matplotlib.

Notes

1. This function can plot the curve of only one target variable. If not specified, index 0 is ploted by default.

2. This function supports ploting of historical observations, future actual values, and forecast intervals.

Parameters:

• forecast ('DataFrame'. The columns need to include the timestamp column) – and the target columns.

• timestamp_col (str. 'timestamp' column name.) –

• target_col (str or list. target columns name.) –

• var_id ('int' or 'str'. If int, it is the index of the target column. If str,) – it is the name of the target column. default 0.

• actual ('DataFrame' or None. If it is not None, the column needs to include) – the time column and the target column.

• history ('DataFrame'. History data. The columns need to include the timestamp column) – and the target columns.

• forecast_interval ('DataFrame'. Forecast confidence interval.) –

• show_forecast_interval ('bool'. Whether to show the forecast intervals.) – Default False.

• include_history ('bool'. Whether to show the historical observations.) – Default True.

• figsize ((float, float), figure.figsize Width, height in inches.) – Default (16, 6).

• grid ('bool'. Whether to display the grid.) – Default True.

Returns:

fig

Return type:

‘matpltlib..pyplot.figure’.

utils.transformers

class hyperts.utils.transformers.TimeSeriesTransformer(time_series_col=None)

Bases: object

Scale time series features.

transform(X, y=None, **kwargs)

fit(X, y=None, **kwargs)

class hyperts.utils.transformers.LogXplus1Transformer(nan_tolerance=5, eps=1e-08, copy=True)

Bases: BaseEstimator, TransformerMixin

Scale each feature by log(x+1).

Parameters:

• nan_tolerance (int, default=5.) – Tolerate the number of nans that exist.

• eps (float, default=1e-8.) – To prevent the division by 0.

• copy (bool, default=True.) – Set to False to perform inplace row normalization and avoid a copy.

fit(X, y=None, **kwargs)

transform(X, y=None, **kwargs)

inverse_transform(X, y=None, **kwargs)

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Input samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs), default=None) – Target values (None for unsupervised transformations).

• **fit_params (dict) – Additional fit parameters.

Returns:

X_new – Transformed array.

Return type:

ndarray array of shape (n_samples, n_features_new)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class hyperts.utils.transformers.IdentityTransformer

Bases: BaseEstimator, TransformerMixin

Identity transformer.

fit(X, y=None, **kwargs)

transform(X, y=None, **kwargs)

inverse_transform(X, y=None, **kwargs)

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Input samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs), default=None) – Target values (None for unsupervised transformations).

• **fit_params (dict) – Additional fit parameters.

Returns:

X_new – Transformed array.

Return type:

ndarray array of shape (n_samples, n_features_new)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class hyperts.utils.transformers.StandardTransformer(eps=1e-08, copy=True)

Bases: BaseEstimator, TransformerMixin

Standardize features by removing the mean and scaling to unit variance.

Notes

Unlike scikit-learn, it can process 3D time series - (nb_samples, series_length, nb_dims).

The transformation is given by:

X_scaled = (X - X.mean) / (X.var + eps),

where, for 2D features:

mean = X.mean(axis=0), var = ((X - mean) ** 2).mean(axis=0)

for 3D features:

mean = X.mean(axis=0, keepdims=True).mean(axis=1, keepdims=True), var = ((X - mean) ** 2).mean(axis=0, keepdims=True).mean(axis=1, keepdims=True).

Parameters:

• eps (float, default=1e-8.) – To prevent the division by 0.

• copy (bool, default=True.) – Set to False to perform inplace row normalization and avoid a copy.

fit(X, y=None, **kwargs)

transform(X, y=None, **kwargs)

inverse_transform(X, y=None, **kwargs)

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Input samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs), default=None) – Target values (None for unsupervised transformations).

• **fit_params (dict) – Additional fit parameters.

Returns:

X_new – Transformed array.

Return type:

ndarray array of shape (n_samples, n_features_new)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class hyperts.utils.transformers.MinMaxTransformer(eps=1e-08, copy=True)

Bases: BaseEstimator, TransformerMixin

Transform features by scaling each feature to a given range.

Notes

Unlike scikit-learn, it can process 3D time series - (nb_samples, series_length, nb_dims).

The transformation is given by:

X_scaled = (X - X.min) / (X.max - X.min + eps),

where, for 2D features:

min = X.min(axis=0, initial=None), max = X.max(axis=0, initial=None),

for 3D features:

min = X.min(axis=0, keepdims=True, initial=None).min(axis=1, keepdims=True), max = X.max(axis=0, keepdims=True, initial=None).max(axis=1, keepdims=True).

Parameters:

• eps (float, default=1e-8.) – To prevent the division by 0.

• copy (bool, default=True.) – Set to False to perform inplace row normalization and avoid a copy.

fit(X, y=None, **kwargs)

transform(X, y=None, **kwargs)

inverse_transform(X, y=None, **kwargs)

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Input samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs), default=None) – Target values (None for unsupervised transformations).

• **fit_params (dict) – Additional fit parameters.

Returns:

X_new – Transformed array.

Return type:

ndarray array of shape (n_samples, n_features_new)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class hyperts.utils.transformers.MaxAbsTransformer(eps=1e-08, copy=True)

Bases: BaseEstimator, TransformerMixin

Scale each feature by its maximum absolute value.

Notes

Unlike scikit-learn, it can process 3D time series - (nb_samples, series_length, nb_dims).

The transformation is given by:

X_scaled = X / (X.max_abs + eps),

where, for 2D features:

max_abs = np.max(np.abs(X), axis=0),

for 3D features:

max_abs = np.abs(X).max(axis=0, keepdims=True).max(axis=1, keepdims=True)

Parameters:

• eps (float, default=1e-8.) – To prevent the division by 0.

• copy (bool, default=True.) – Set to False to perform inplace row normalization and avoid a copy.

fit(X, y=None, **kwargs)

transform(X, y=None, **kwargs)

inverse_transform(X, y=None, **kwargs)

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Input samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs), default=None) – Target values (None for unsupervised transformations).

• **fit_params (dict) – Additional fit parameters.

Returns:

X_new – Transformed array.

Return type:

ndarray array of shape (n_samples, n_features_new)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class hyperts.utils.transformers.CategoricalTransformer(label_encoder=None, onehot_encoder=None, copy=True)

Bases: BaseEstimator, TransformerMixin

Transform categorical labels to one hot labels.

Parameters:

• label_encoder (An existing Label encoder, default=None.) –

• onehot_encoder (An existing OneHot encoder, default=None.) –

• copy (bool, default=True.) – Set to False to perform inplace row normalization and avoid a copy.

fit(X, y=None, **kwargs)

transform(X, y=None, **kwargs)

inverse_transform(X, y=None, **kwargs)

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Input samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs), default=None) – Target values (None for unsupervised transformations).

• **fit_params (dict) – Additional fit parameters.

Returns:

X_new – Transformed array.

Return type:

ndarray array of shape (n_samples, n_features_new)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class hyperts.utils.transformers.CovariateTransformer(covariables, data_cleaner_args=None)

Bases: BaseEstimator, TransformerMixin

Transform covariates by ‘drop_constant_columns’, ‘drop_duplicated_columns’,

‘drop_idness_columns’, ‘replace_inf_values’ and so on.

Parameters:

• covariables (list[n*str], if the data contains covariables, specify the) – covariable column names, (default=None).

• data_cleaner_args (dict or None, (default=None).) –

  If not None, the definition example is as follows:

  data_cleaner_args = {

  ‘correct_object_dtype’: False, ‘int_convert_to’: ‘str’, ‘drop_constant_columns’: True, ‘drop_duplicated_columns’: True, ‘drop_idness_columns’: True, ‘replace_inf_values’: np.nan, …

  }

• details (Reference for) –

fit(X, y=None, **kwargs)

transform(X, y=None, **kwargs)

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Input samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs), default=None) – Target values (None for unsupervised transformations).

• **fit_params (dict) – Additional fit parameters.

Returns:

X_new – Transformed array.

Return type:

ndarray array of shape (n_samples, n_features_new)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class hyperts.utils.transformers.OutliersTransformer(method='clip', std_threshold=3, freq=None, copy=True)

Bases: BaseEstimator, TransformerMixin

Remove outliers.

Parameters:

• method (str, 'clip' or 'fill'.) –

• std_threshold (int, the threshold of std.) –

• freq (str, DateOffset or None, default None.) –

• copy (bool, default=True.) – Set to False to perform inplace row normalization and avoid a copy.

fit(X, y=None, **kwargs)

transform(X, y=None, **kwargs)

inverse_transform(X, y=None, **kwargs)

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Input samples.

• y (array-like of shape (n_samples,) or (n_samples, n_outputs), default=None) – Target values (None for unsupervised transformations).

• **fit_params (dict) – Additional fit parameters.

Returns:

X_new – Transformed array.

Return type:

ndarray array of shape (n_samples, n_features_new)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class hyperts.utils.transformers.TimeSeriesHyperTransformer(space=None, name=None, **kwargs)

Bases: HyperTransformer

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.utils.transformers.LogXplus1HyperTransformer(space=None, name=None, **kwargs)

Bases: HyperTransformer

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

utils.tstoolbox

class hyperts.utils.tstoolbox.TSToolBox

Bases: ToolBox

static DataFrame(data=None, index=None, columns=None, dtype=None, copy=False)

Two-dimensional, size-mutable, potentially heterogeneous tabular data.

Parameters:

• data (ndarray (structured or homogeneous), Iterable, dict, or DataFrame) –

  Dict can contain Series, arrays, constants, or list-like objects.

  Changed in version 0.23.0.

  If data is a dict, column order follows insertion-order for Python 3.6 and later.

  Changed in version 0.25.0.

  If data is a list of dicts, column order follows insertion-order for Python 3.6 and later.

• index (Index or array-like) – Index to use for resulting frame. Will default to RangeIndex if no indexing information part of input data and no index provided.

• columns (Index or array-like) – Column labels to use for resulting frame. Will default to RangeIndex (0, 1, 2, …, n) if no column labels are provided.

• dtype (dtype, default None) – Data type to force. Only a single dtype is allowed. If None, infer.

• copy (bool, default False) – Copy data from inputs. Only affects DataFrame / 2d ndarray input.

static join_df(df1: DataFrame, df2: DataFrame, on: None)

Join columns of another DataFrame.

Parameters:

on (str, list of str, or array-like, optional) – Column or index level name(s) in the caller to join on the index in other, otherwise joins index-on-index. If multiple values given, the other DataFrame must have a MultiIndex. Can pass an array as the join key if it is not already contained in the calling DataFrame. Like an Excel VLOOKUP operation.

Returns:

A dataframe containing columns from both the caller and other.

Return type:

DataFrame

static to_datetime(df: DataFrame, **kwargs)

Convert argument to datetime.

static date_range(start=None, end=None, periods=None, freq=None, **kwargs)

Return a fixed frequency DatetimeIndex.

Parameters:

• start (str or datetime-like, optional) – Left bound for generating dates.

• end (str or datetime-like, optional) – Right bound for generating dates.

• periods (int, optional) – Number of periods to generate.

• freq (str or DateOffset, default 'D') – Frequency strings can have multiples, e.g. ‘5H’. See here for a list of frequency aliases.

static datetime_format(df: DataFrame, format='%Y-%m-%d %H:%M:%S')

Convert datetime format.

static select_1d_forward(arr, indices)

Select by indices from the first axis(0) with forward.

static select_1d_reverse(arr, indices)

Select by indices from the first axis(0) with reverse.

static columns_values(df: DataFrame)

Get column values.

static sort_values(df: DataFrame, ts_name: str = 'timestamp')

Sort in time order.

static drop(df: DataFrame, labels=None, index=None, columns=None, axis: int = 0, inplace: bool = False)

Drop specified labels from rows or columns.

static pop(df: DataFrame, item)

Return item and drop from frame. Raise KeyError if not found.

static columns_tolist(df: DataFrame)

Return a list of the DataFrame columns.

static arange(*args)

Return evenly spaced values within a given interval.

static infer_ts_freq(df: DataFrame, ts_name: str = 'timestamp')

Infer the frequency of the time series. :param ts_name: :type ts_name: ‘str’, time column name.

static multi_period_loop_imputer(df: DataFrame, freq: str, offsets: Optional[list] = None, max_loops: int = 10)

Multiple period loop impute NAN. :param freq: ‘S’ - second

‘T’ - minute ‘H’ - hour ‘D’ - day ‘M’ - month ‘Y’,’A’, A-DEC’ - year

Parameters:

• offsets (list, offset lag.) –

• max_loops ('int', maximum number of loop imputed.) –

static forward_period_imputer(df: DataFrame, offset: int)

Forward period imputer. :param offsets: :type offsets: ‘int’, offset lag.

static simple_numerical_imputer(df: DataFrame, mode='mean')

Fill NaN with mean, mode, 0.

static drop_duplicated_ts_rows(df: DataFrame, ts_name: str = 'timestamp', keep_data: str = 'last')

Returns without duplicate time series, the last be keeped by default. .. rubric:: Example

TimeStamp y 2021-03-01 3.4 2021-03-02 5.2 2021-03-03 9.3 2021-03-03 9.5 2021-03-04 6.7 2021-03-05 2.3 >> TimeStamp y 2021-03-01 3.4 2021-03-02 5.2 2021-03-03 9.5 2021-03-04 6.7 2021-03-05 2.3

static smooth_missed_ts_rows(df: DataFrame, freq: Optional[str] = None, ts_name: str = 'timestamp')

Returns full time series. .. rubric:: Example

TimeStamp y 2021-03-01 3.4 2021-03-02 5.2 2021-03-04 6.7 2021-03-05 2.3 >> TimeStamp y 2021-03-01 3.4 2021-03-02 5.2 2021-03-03 NaN 2021-03-04 6.7 2021-03-05 2.3

static clip_to_outliers(df, std_threshold: int = 3)

Replace outliers above threshold with that threshold. :param std_threshold: :type std_threshold: ‘float’, the number of standard deviations away from mean to count as outlier.

static nan_to_outliers(df, std_threshold: int = 3)

Replace outliers above threshold with that threshold. :param std_threshold: :type std_threshold: ‘float’, the number of standard deviations away from mean to count as outlier.

static infer_window_size(max_size: int, freq: str)

Infer window of neural net. :param max_size: :type max_size: int, maximum time window allowed. :param freq: :type freq: str or DateOffset.

static fft_infer_period(data)

Fourier inference period.

References

https://github.com/xuawai/AutoPeriod/blob/master/auto_period.ipynb

static generate_time_covariates(start_date, periods, freq='H')

Generate covariates about time.

Parameters:

• start_date ('str' or datetime-like.) – Left bound for generating dates.

• periods ('int'.) – Number of periods to generate.

• freq (str or DateOffset, default 'H'.) –

static df_mean_std(data: DataFrame)

Get the mean and standard deviation of the data.

static infer_forecast_interval(forecast, prior_mu, prior_sigma, n: int = 5, confidence_level: float = 0.9)

A corruption of Bayes theorem. It will be sensitive to the transformations of the data.

static from_3d_array_to_nested_df(data: ndarray, columns: Optional[str] = None, cells_as_array: bool = False)

Convert Numpy ndarray with shape (nb_samples, series_length, nb_variables) into nested pandas DataFrame (with time series as numpy array or pandas Series in cells)

Parameters:

• data (np.ndarray) – 3-dimensional Numpy array to convert to nested pandas DataFrame format

• columns (list-like, default = None) – Optional list of names to use for naming nested DataFrame’s columns

• cells_as_array (bool, default = False) – If True, then nested cells contain Numpy array If False, then nested cells contain pandas Series

Returns:

df

Return type:

pd.DataFrame

References

sktime_data_processing: https://github.com/Riyabelle25/sktime/blob/main/sktime/utils/data_processing.py

static from_nested_df_to_3d_array(data: DataFrame)

Convert nested pandas DataFrame (with time series as numpy array or pandas Series in cells) into Numpy ndarray with shape (nb_samples, series_length, nb_variables).

Parameters:

data (pd.DataFrame) – Nested pandas DataFrame

Returns:

• data_3d (np.arrray) – 3-dimensional NumPy array

• References

• ———-from_nested_to_3d_numpy

• sktime_data_processing (https://github.com/Riyabelle25/sktime/blob/main/sktime/utils/data_processing.py)

static is_nested_dataframe(data: DataFrame)

Determines whether data is a nested Dataframe.

Returns:

bool

Return type:

True or False.

static random_train_test_split(*arrays, test_size=None, train_size=None, random_state=None, shuffle=True, stratify=None)

Split arrays or matrices into random train and test subsets. This is a wrapper of scikit-learn’s train_test_split that has shuffle.

static temporal_train_test_split(*arrays, test_size=None, train_size=None, test_horizon=None)

Split arrays or matrices into sequential train and test subsets.This is a wrapper of scikit-learn’s train_test_split that does not shuffle.

Parameters:

• *arrays (sequence of indexables with same length / shape[0] Allowed inputs) –

• lists (are) –

• arrays (numpy) –

• dataframes. (scipy-sparse matrices or pandas) –

• test_size (float, int or None, optional (default=None)) – If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it will be set to 0.25.

• train_size (float, int, or None, (default=None)) – If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int, represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.

• test_horizon (int or None, (default=None)) – If int, represents the forecast horizon length.

Returns:

splitting – List containing train-test split of inputs.

Return type:

list, length=2 * len(arrays)

static list_diff(p: list, q: list)

Gets the difference set of two lists. :param p: :type p: list. :param q: :type q: list. :param Returns: :param A list.: :param ——-: :param Example: p = [1, 2, 3, 4, 5], q = [2, 4]

>> list_diff(p, q) >> [1, 3, 5]

p = [1, 2, 3, 4, 5], q = [] >> list_diff(p, q) >> [1, 2, 3, 4, 5]

static infer_pos_label(y_true, task, label_name=None, pos_label=None)

metrics

alias of Metrics

classmethod preqfold(strategy='preq-bls', base_size=None, n_splits=5, stride=1, *, max_train_size=None, test_size=None, gap_size=0)

classmethod greedy_ensemble(task, estimators, need_fit=False, n_folds=5, method='soft', random_state=9527, target_dims=1, scoring='neg_log_loss', ensemble_size=0)

STRATEGY_NUMBER = 'number'

STRATEGY_QUANTILE = 'quantile'

STRATEGY_THRESHOLD = 'threshold'

classmethod accept(*args)

acceptable_types = (<class 'numpy.ndarray'>, <class 'pandas.core.frame.DataFrame'>, <class 'pandas.core.series.Series'>)

static array_to_df(arr, *, columns=None, index=None, meta=None)

static collapse_last_dim(arr, keep_dim=True)

Collapse the last dimension :param arr: data array :param keep_dim: keep the last dim as one or not :return:

classmethod collinearity_detector()

column_selector = <module 'hypernets.tabular.column_selector' from '/home/docs/checkouts/readthedocs.org/user_builds/hyperts/envs/latest/lib/python3.7/site-packages/hypernets/tabular/column_selector.py'>

compute_class_weight(*, classes, y)

Estimate class weights for unbalanced datasets.

Parameters:

• class_weight (dict, 'balanced' or None) – If ‘balanced’, class weights will be given by n_samples / (n_classes * np.bincount(y)). If a dictionary is given, keys are classes and values are corresponding class weights. If None is given, the class weights will be uniform.

• classes (ndarray) – Array of the classes occurring in the data, as given by np.unique(y_org) with y_org the original class labels.

• y (array-like of shape (n_samples,)) – Array of original class labels per sample.

Returns:

class_weight_vect – Array with class_weight_vect[i] the weight for i-th class.

Return type:

ndarray of shape (n_classes,)

References

The “balanced” heuristic is inspired by Logistic Regression in Rare Events Data, King, Zen, 2001.

static compute_sample_weight(y)

static concat_df(dfs, axis=0, repartition=False, random_state=9527, **kwargs)

classmethod data_cleaner(nan_chars=None, correct_object_dtype=True, drop_constant_columns=True, drop_duplicated_columns=False, drop_label_nan_rows=True, drop_idness_columns=True, replace_inf_values=nan, drop_columns=None, reserve_columns=None, reduce_mem_usage=False, int_convert_to='float')

classmethod data_hasher(method='md5')

static detect_strategy(strategy, *, threshold=None, quantile=None, number=None, default_strategy, default_threshold, default_quantile, default_number)

classmethod detect_strategy_of_feature_selection_by_importance(strategy, *, threshold=None, quantile=None, number=None)

static df_to_array(df)

classmethod drift_detector(preprocessor=None, estimator=None, random_state=None)

classmethod estimator_detector(name_or_cls, task, *, init_kwargs=None, fit_kwargs=None, n_samples=100, n_features=5)

classmethod feature_selector_with_drift_detection(remove_shift_variable=True, variable_shift_threshold=0.7, variable_shift_scorer=None, auc_threshold=0.55, min_features=10, remove_size=0.1, sample_balance=True, max_test_samples=None, cv=5, random_state=None, callbacks=None)

classmethod feature_selector_with_feature_importances(strategy=None, threshold=None, quantile=None, number=None)

static fix_binary_predict_proba_result(proba)

static from_local(*data)

static gc()

classmethod general_estimator(X, y=None, estimator=None, task=None)

classmethod general_preprocessor(X, y=None)

static get_shape(X, allow_none=False)

classmethod hstack_array(arrs)

classmethod infer_task_type(y, excludes=None)

classmethod kfold(n_splits=5, *, shuffle=False, random_state=None)

static load_data(data_path, *, reset_index=False, reader_mapping=None, **kwargs)

static mean_oof(probas)

static memory_free()

static memory_total()

static memory_usage(*data)

static merge_oof(oofs)

Parameters:

oofs – list of tuple(idx,proba)

Returns:

merged proba

static nunique_df(df)

static parquet()

static permutation_importance(estimator, X, y, *, scoring=None, n_repeats=5, n_jobs=None, random_state=None, sample_weight=None, max_samples=1.0)

see: sklearn.inspection.permutation_importance

classmethod permutation_importance_batch(estimators, X, y, scoring=None, n_repeats=5, n_jobs=None, random_state=None)

Evaluate the importance of features of a set of estimators

Parameters:

• estimators (list) – A set of estimators that has already been fitted and is compatible with scorer.

• X (ndarray or DataFrame, shape (n_samples, n_features)) – Data on which permutation importance will be computed.

• y (array-like or None, shape (n_samples, ) or (n_samples, n_classes)) – Targets for supervised or None for unsupervised.

• scoring (string, callable or None, default=None) – Scorer to use. It can be a single string (see scoring_parameter) or a callable (see scoring). If None, the estimator’s default scorer is used.

• n_repeats (int, default=5) – Number of times to permute a feature.

• n_jobs (int or None, default=None) – The number of jobs to use for the computation. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.

• random_state (int, RandomState instance, or None, default=None) – Pseudo-random number generator to control the permutations of each feature. See random_state.

Returns:

result – Dictionary-like object, with attributes:

importances_meanndarray, shape (n_features, )

Mean of feature importance over n_repeats.

importances_stdndarray, shape (n_features, )

Standard deviation over n_repeats.

importancesndarray, shape (n_features, n_repeats)

Raw permutation importance scores.

Return type:

Bunch

classmethod pseudo_labeling(strategy, threshold=None, quantile=None, number=None)

static reset_index(df)

static select_1d(arr, indices)

Select by indices from the first axis(0).

static select_df(df, indices)

Select dataframe by row indices.

classmethod select_feature_by_importance(feature_importance, strategy=None, threshold=None, quantile=None, number=None)

static select_valid_oof(y, oof)

static stack_array(arrs, axis=0)

classmethod statified_kfold(n_splits=5, *, shuffle=False, random_state=None)

static take_array(arr, indices, axis=None)

static to_local(*data)

train_test_split(*, test_size=None, train_size=None, random_state=None, shuffle=True, stratify=None)

Split arrays or matrices into random train and test subsets.

Quick utility that wraps input validation and next(ShuffleSplit().split(X, y)) and application to input data into a single call for splitting (and optionally subsampling) data in a oneliner.

Read more in the User Guide.

Parameters:

• *arrays (sequence of indexables with same length / shape[0]) – Allowed inputs are lists, numpy arrays, scipy-sparse matrices or pandas dataframes.

• test_size (float or int, default=None) – If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it will be set to 0.25.

• train_size (float or int, default=None) – If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int, represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.

• random_state (int, RandomState instance or None, default=None) – Controls the shuffling applied to the data before applying the split. Pass an int for reproducible output across multiple function calls. See Glossary.

• shuffle (bool, default=True) – Whether or not to shuffle the data before splitting. If shuffle=False then stratify must be None.

• stratify (array-like, default=None) – If not None, data is split in a stratified fashion, using this as the class labels. Read more in the User Guide.

Returns:

splitting – List containing train-test split of inputs.

New in version 0.16: If the input is sparse, the output will be a scipy.sparse.csr_matrix. Else, output type is the same as the input type.

Return type:

list, length=2 * len(arrays)

Examples

>>> import numpy as np
>>> from sklearn.model_selection import train_test_split
>>> X, y = np.arange(10).reshape((5, 2)), range(5)
>>> X
array([[0, 1],
       [2, 3],
       [4, 5],
       [6, 7],
       [8, 9]])
>>> list(y)
[0, 1, 2, 3, 4]

>>> X_train, X_test, y_train, y_test = train_test_split(
...     X, y, test_size=0.33, random_state=42)
...
>>> X_train
array([[4, 5],
       [0, 1],
       [6, 7]])
>>> y_train
[2, 0, 3]
>>> X_test
array([[2, 3],
       [8, 9]])
>>> y_test
[1, 4]

>>> train_test_split(y, shuffle=False)
[[0, 1, 2], [3, 4]]

static unique(y)

static value_counts(ar)

classmethod vstack_array(arrs)

hyperts.framework

framework.compete

class hyperts.framework.compete.TSAFDataPreprocessStep(experiment, name, timestamp_col=None, freq=None, cv=False, covariate_cols=None, covariate_cleaner=None, ensemble_size=None, train_data_periods=None, anomaly_label_col=None, contamination=0.05)

Bases: ExperimentStep

Time Series Anomaly Detection or Forecasting Task Data Preprocess Step.

fit_transform(hyper_model, X_train, y_train, X_test=None, X_eval=None, y_eval=None, **kwargs)

transform(X, y=None, **kwargs)

covariate_transform(X)

series_transform(X, target_cols=None)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

get_fitted_params()

STATUS_FAILED = 1

STATUS_NONE = -1

STATUS_RUNNING = 10

STATUS_SKIPPED = 2

STATUS_SUCCESS = 0

property elapsed_seconds

is_transform_skipped()

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

step_progress(*args, **kwargs)

property task

class hyperts.framework.compete.TSRCDataPreprocessStep(experiment, name, cv=False)

Bases: ExperimentStep

Time Series Classification or Regression Task Data Preprocess Step.

fit_transform(hyper_model, X_train, y_train, X_test=None, X_eval=None, y_eval=None, **kwargs)

transform(X, y=None, **kwargs)

panel_transform(X, y=None)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

get_fitted_params()

STATUS_FAILED = 1

STATUS_NONE = -1

STATUS_RUNNING = 10

STATUS_SKIPPED = 2

STATUS_SUCCESS = 0

property elapsed_seconds

is_transform_skipped()

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

step_progress(*args, **kwargs)

property task

class hyperts.framework.compete.TSSpaceSearchStep(experiment, name, cv=False, num_folds=3)

Bases: SpaceSearchStep

Time Series Space Searching.

search(X_train, y_train, X_test=None, X_eval=None, y_eval=None, **kwargs)

STATUS_FAILED = 1

STATUS_NONE = -1

STATUS_RUNNING = 10

STATUS_SKIPPED = 2

STATUS_SUCCESS = 0

property elapsed_seconds

estimate_time_limit(total_time_limit)

static find_early_stopping_callback(cbs)

fit_transform(hyper_model, X_train, y_train, X_test=None, X_eval=None, y_eval=None, **kwargs)

from_fitted_step(fitted_step)

get_fitted_params()

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

is_transform_skipped()

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

step_progress(*args, **kwargs)

property task

transform(X, y=None, **kwargs)

class hyperts.framework.compete.TSEnsembleStep(experiment, name, mode=None, scorer=None, ensemble_size=10, cv=False, retrain_on_wholedata=False)

Bases: EnsembleStep

Time Series Ensemble.

build_estimator(hyper_model, X_train, y_train, X_eval=None, y_eval=None, **kwargs)

get_ensemble(estimators, X_train, y_train)

est_retrain(trials, hyper_model, X_train, y_train, X_eval=None, y_eval=None, weights=None, **kwargs)

STATUS_FAILED = 1

STATUS_NONE = -1

STATUS_RUNNING = 10

STATUS_SKIPPED = 2

STATUS_SUCCESS = 0

property elapsed_seconds

fit_transform(hyper_model, X_train, y_train, X_test=None, X_eval=None, y_eval=None, **kwargs)

get_ensemble_predictions(trials, ensemble)

get_fitted_params()

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

is_transform_skipped()

select_trials(hyper_model)

select trails to ensemble from hyper_model (and it’s history)

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

step_progress(*args, **kwargs)

property task

transform(X, y=None, **kwargs)

class hyperts.framework.compete.TSFinalTrainStep(experiment, name, mode=None, retrain_on_wholedata=False)

Bases: FinalTrainStep

build_estimator(hyper_model, X_train, y_train, X_test=None, X_eval=None, y_eval=None, **kwargs)

STATUS_FAILED = 1

STATUS_NONE = -1

STATUS_RUNNING = 10

STATUS_SKIPPED = 2

STATUS_SUCCESS = 0

property elapsed_seconds

fit_transform(hyper_model, X_train, y_train, X_test=None, X_eval=None, y_eval=None, **kwargs)

get_fitted_params()

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

is_transform_skipped()

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

step_progress(*args, **kwargs)

property task

transform(X, y=None, **kwargs)

class hyperts.framework.compete.TSPipeline(sk_pipeline, freq, task, mode, timestamp, covariables, target, history=None, **kwargs)

Bases: object

Pipeline Extension for Time Series Analysis.

Parameters:

• sk_pipeline (sklearn pipeline, including data_preprocessing, space_searching, final_training and estimator) – steps and so on.

• freq ('str', DateOffset or None, default None.) –

• task ('str' or None, default None.) – Task could be ‘univariate-forecast’, ‘multivariate-forecast’, and ‘univariate-binaryclass’, etc. See consts.py for details.

• mode (str, default 'stats'. Optional {'stats', 'dl', 'nas'}, where,) – ‘stats’ indicates that all the models selected in the execution experiment are statistical models. ‘dl’ indicates that all the models selected in the execution experiment are deep learning models. ‘nas’ indicates that the selected model of the execution experiment will be a deep network model for neural architecture search, which is not currently supported.

• timestamp (str, forecast task 'timestamp' cannot be None, (default=None).) –

• covariables (list[n*str], if the data contains covariables, specify the covariable column names, (default=None).) –

• target (str or list, optional.) – Target feature name for training, which must be one of the train_data columns for classification[str], regression[str] or unvariate forecast task [list]. For multivariate forecast task, it is multiple columns of training data.

predict(X, forecast_start=None)

Predicts target for sequences in X.

Parameters:

• X ('DataFrame'.) – For forecast task, X.columns = [‘timestamp’, (covariate_1), (covariate_2),…]. (covariate_1) indicates that it may not exist. For classification or regression tasks, X.columns = [variate_1, variate_2,…].

• forecast_start ('DataFrame'. This parameter applies only to 'dl' mode.) – Forecast the start fragment, if None, by default the last window fragment of the train data. forecast_start.columns = [‘timestamp’, variate_1, variate_2, …, (covariate_1), (covariate_2),…]. (covariate_1) indicates that it may not exist.

predict_proba(X)

Predicts target probabilities for sequences in X for classification task.

Parameters:

X ('DataFrame'.) – X.columns = [variate_1, variate_2,…].

evaluate(y_true, y_pred, y_proba=None, metrics=None)

Evaluates model performance.

Parameters:

• y_true ('np.arrray'.) –

• y_pred ('pd.DataFrame' or 'np.arrray'.) – For forecast task, ‘pd.DataFrame’, X.columns could be [‘timestamp’, (covariate_1), (covariate_2),…, variate_1, variate_2,…]. (covariate_1) indicates that it may not exist. For classification and regression tasks, ‘np.arrray’.

• y_proba ('np.arrray' or None, some metrics should be given, such as AUC.) –

• metrics (list, tuple or None. If None,) – For forecast or regression tasks, metrics = [‘mae’, ‘mse’, ‘rmse’, ‘mape’, ‘smape’], For classification task, metrics = [‘accuracy’, ‘f1’, ‘precision’, ‘recall’].

make_future_dataframe(periods=None, start_date=None, covariate_df=None)

Simulate the trend using the extrapolated generative model.

Notes

If covariate_df exists, the start time of the covariate_df must be the same as that of start_date. In addition, the covariate_df length is not less than periods.

Parameters:

• periods ('int', number of periods to forecast forward.) –

• start_date ('str' or datetime-like, forecast the future start date.) –

Returns:

• pd.Dataframe that extends forward from the end of history for the

• requested number of periods.

plot(forecast, actual=None, history=None, var_id=0, show_forecast_interval=True, interactive=True, figsize=None)

Plots forecast trend curves for the forecst task.

Notes

1. This function can plot the curve of only one target variable. If not specified, index 0 is ploted by default.

2. This function supports ploting of historical observations, future actual values, and forecast intervals.

Parameters:

• forecast ('DataFrame'. The columns need to include the timestamp column) – and the target columns.

• actual ('DataFrame' or None. If it is not None, the column needs to include) – the time column and the target column.

• var_id ('int' or 'str'. If int, it is the index of the target column. If str,) – it is the name of the target column. default 0.

• show_forecast_interval ('bool'. Whether to show the forecast intervals.) – Default True.

Returns:

fig

Return type:

‘plotly.graph_objects.Figure’.

forecast_interval(forecast, confidence_level=0.9)

infer forecast interval.

Parameters:

• forecast ('DataFrame'. The columns need to include the timestamp column) – and the target columns.

• confidence_level (float, default 0.9.) –

split_X_y(data, smooth=False, impute=False)

Splits the data into X and y.

Parameters:

• data ('DataFrame', including X and y.) –

• smooth (Whether to smooth missed time series rows. Default False.) –

  Example:

  TimeStamp y 2021-03-01 3.4 2021-03-02 5.2 2021-03-04 6.7 2021-03-05 2.3 >> TimeStamp y 2021-03-01 3.4 2021-03-02 5.2 2021-03-03 NaN 2021-03-04 6.7 2021-03-05 2.3

• impute (Whether to impute in missing values. Default False.) –

  Example:

  TimeStamp y 2021-03-01 3.4 2021-03-02 5.2 2021-03-03 NaN 2021-03-04 6.7 2021-03-05 2.3 >> TimeStamp y 2021-03-01 3.4 2021-03-02 5.2 2021-03-03 3.4 2021-03-04 6.7 2021-03-05 2.3

Return type:

X, y.

get_pipeline_params()

Gets sklearn pipeline parameters.

save(model_file)

Save TSPipeline.

Parameters:

model_file (str, the path where the file is saved. For example, "home/xxx/xxx/models".) –

static load(model_file)

Load TSPipeline.

Parameters:

model_file (str, the path where the file is saved. For example,) –

1. “home/xxx/xxx/models/stats_models”

2. ”home/xxx/xxx/models/dl_models”

3. ”home/xxx/xxx/models/nas_models”

class hyperts.framework.compete.TSCompeteExperiment(hyper_model, X_train, y_train, X_eval=None, y_eval=None, X_test=None, eval_size=0.2, freq=None, target_col=None, timestamp_col=None, covariate_cols=None, anomaly_label_col=None, covariate_cleaner=None, cv=False, num_folds=3, task=None, mode='stats', id=None, forecast_train_data_periods=None, hist_store_upper_limit=200, contamination=0.05, callbacks=None, log_level=None, random_state=None, scorer=None, optimize_direction=None, ensemble_size=10, final_retrain_on_wholedata=True, **kwargs)

Bases: SteppedExperiment

A powerful experiment strategy for Automatic Time Series with a set of advanced features.

Parameters:

• hyper_model (hypernets.model.HyperModel) – A HyperModel instance

• X_train (Pandas or Dask DataFrame) – Feature data for training

• y_train (Pandas or Dask Series) – Target values for training

• X_eval ((Pandas or Dask DataFrame) or None) – (default=None), Feature data for evaluation

• y_eval ((Pandas or Dask Series) or None, default None.) – Target values for evaluation

• X_test ((Pandas or Dask Series) or None, default None.) – Unseen data without target values for semi-supervised learning

• eval_size ('float' or 'int', default None.) – Only valid when X_eval or y_eval is None. If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the eval split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size.

• freq ('str', DateOffset or None, default None.) –

• target_col ('str' or list[str], default None.) –

• timestamp_col (str or None, default None.) –

• covariate_cols (list[list or None, list or None] or None, default None. covariate_cols needs to contain) – a list of original covariates and a list of cleaned covariates.

• covariate_data_cleaner_args ('dict' or None, default None. Suitable for forecast task.) – Dictionary of parameters to initialize the DataCleaner instance. If None, DataCleaner will initialized with default values.

• data_cleaner_args ('dict' or None, default None. Suitable for classification/regression tasks.) – Dictionary of parameters to initialize the DataCleaner instance. If None, DataCleaner will initialized with default values.

• cv ('bool', default False.) – If True, use cross-validation instead of evaluation set reward to guide the search process.

• num_folds ('int', default 3.) – Number of cross-validated folds, only valid when cv is true.

• task ('str' or None, default None.) – Task could be ‘univariate-forecast’, ‘multivariate-forecast’, and ‘univariate-binaryclass’, etc. See consts.py for details.

• mode (str, default 'stats'. Optional {'stats', 'dl', 'nas'}, where,) – ‘stats’ indicates that all the models selected in the execution experiment are statistical models. ‘dl’ indicates that all the models selected in the execution experiment are deep learning models. ‘nas’ indicates that the selected model of the execution experiment will be a deep network model for neural architecture search, which is not currently supported.

• id (trial id, default None.) –

• callbacks (list of callback functions or None, default None.) – List of callback functions that are applied at each experiment step. See hypernets.experiment.ExperimentCallback for more information.

• log_level ('int', 'str', or None, default None,) –

  Level of logging, possible values:

  -logging.CRITICAL -logging.FATAL -logging.ERROR -logging.WARNING -logging.WARN -logging.INFO -logging.DEBUG -logging.NOTSET

• random_state ('int' or RandomState instance, default None.) – Controls the shuffling applied to the data before applying the split.

• scorer ('str', callable or None, default None.) – Scorer to used for feature importance evaluation and ensemble. It can be a single string (see [get_scorer](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.get_scorer.html)) or a callable (see [make_scorer](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html)). Will be inferred from hyper_model.reward_metric if it’s None.

• ensemble_size ('int', default 10.) – The number of estimator to ensemble. During the AutoML process, a lot of models will be generated with different preprocessing pipelines, different models, and different hyperparameters. Usually selecting some of the models that perform well to ensemble can obtain better generalization ability than just selecting the single best model.

• final_retrain_on_wholedata (bool, after the search, whether to retrain the optimal model on the whole data set.) – default True.

run(**kwargs)

Related parameters that can be reset, mainly involving pipeline stap and model parameters.

Including but not limited to the following:

Parameters:

• max_trials (int, maximum number of tests (model search), optional, (default=3).) –

• final_train_epochs (int, the searched best DL model performs the final training epochs, optional, (default=120).) –

• epochs (int, DL model training epochs in each trail, optional, (default=120).) –

• batch_size (int, number of samples per gradient update, optional, (default is self-adaption).) –

• optimizer (str, the optimizer of DL model, optional {'adam', 'sgd', 'rmsprop'}, (default='adam').) –

• learning_rate (float, the optimizer's learning rate, optional, (default=0.001).) –

• loss (str, the loss function of DL model, optional, (default is searching).) –

• drop_rate (float, The rate of Dropout for DL model, optional, (default is searching).) –

• dl_forecast_window (int, the sequence length of each sample in DL model, i.e., timestamp of) – [batch_size, timestamp, dim], optional, (default is searching).

• dl_forecast_length (int, the forecast length for the future in DL model, optional, (default is searching).) –

• seasonality_mode (str, 'additive' (default) or 'multiplicative' in Prophet.) –

• period_offset (int, s of seasonal_order(P,D,Q,s) in ARIMA, optional, (default is searching).) –

• maxlags (int, maximum number of lags to check for order selection in VAR, optional, (default is searching).) –

• ... –

Notes

Once the above parameters are specified, the search space cannot be searched.

to_estimator(X_train, y_train, X_test, X_eval, y_eval, steps)

report_best_trial_params()

Gets experiment best trial parameters.

property elapsed

export_model()

find_step(fn, until_step_name=None, index=False)

get_data_character()

get_step(name)

get_step_index(name_or_index, default)

plot_dataset()

step_break(error=None)

step_end(output=None)

step_progress(progress, eta=None)

step_start(step)

train(hyper_model, X_train, y_train, X_test, X_eval=None, y_eval=None, **kwargs)

Run an experiment

Parameters:

• hyper_model (HyperModel) –

• X_train –

• y_train –

• X_test –

• X_eval –

• y_eval –

framework.estimators

class hyperts.framework.estimators.HyperEstimator(fit_kwargs=None, space=None, name=None, **hyperparams)

Bases: ModuleSpace

An interface class representing a hyperts estimator.

build_estimator(task=None)

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.ProphetForecastEstimator(fit_kwargs=None, growth='linear', changepoints=None, n_changepoints=25, changepoint_range=0.8, yearly_seasonality='auto', weekly_seasonality='auto', daily_seasonality='auto', holidays=None, seasonality_mode='additive', seasonality_prior_scale=10.0, holidays_prior_scale=10.0, changepoint_prior_scale=0.05, mcmc_samples=0, interval_width=0.8, uncertainty_samples=1000, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Forecast Estimator based on Hypernets. Estimator: Prophet. Suitable for: Unvariate Forecast Task.

Parameters:

• growth (String 'linear' or 'logistic' to specify a linear or logistic) – trend.

• changepoints (List of dates at which to include potential changepoints. If) – not specified, potential changepoints are selected automatically.

• n_changepoints (Number of potential changepoints to include. Not used) – if input changepoints is supplied. If changepoints is not supplied, then n_changepoints potential changepoints are selected uniformly from the first changepoint_range proportion of the history.

• changepoint_range (Proportion of history in which trend changepoints will) – be estimated. Defaults to 0.8 for the first 80%. Not used if changepoints is specified.

• yearly_seasonality (Fit yearly seasonality.) – Can be ‘auto’, True, False, or a number of Fourier terms to generate.

• weekly_seasonality (Fit weekly seasonality.) – Can be ‘auto’, True, False, or a number of Fourier terms to generate.

• daily_seasonality (Fit daily seasonality.) – Can be ‘auto’, True, False, or a number of Fourier terms to generate.

• holidays (pd.DataFrame with columns holiday (string) and ds (date type)) – and optionally columns lower_window and upper_window which specify a range of days around the date to be included as holidays. lower_window=-2 will include 2 days prior to the date as holidays. Also optionally can have a column prior_scale specifying the prior scale for that holiday.

• seasonality_mode ('additive' (default) or 'multiplicative'.) –

• seasonality_prior_scale (Parameter modulating the strength of the) – seasonality model. Larger values allow the model to fit larger seasonal fluctuations, smaller values dampen the seasonality. Can be specified for individual seasonalities using add_seasonality.

• holidays_prior_scale (Parameter modulating the strength of the holiday) – components model, unless overridden in the holidays input.

• changepoint_prior_scale (Parameter modulating the flexibility of the) – automatic changepoint selection. Large values will allow many changepoints, small values will allow few changepoints.

• mcmc_samples (Integer, if greater than 0, will do full Bayesian inference) – with the specified number of MCMC samples. If 0, will do MAP estimation.

• interval_width (Float, width of the uncertainty intervals provided) – for the forecast. If mcmc_samples=0, this will be only the uncertainty in the trend using the MAP estimate of the extrapolated generative model. If mcmc.samples>0, this will be integrated over all model parameters, which will include uncertainty in seasonality.

• uncertainty_samples (Number of simulated draws used to estimate) – uncertainty intervals. Settings this value to 0 or False will disable uncertainty estimation and speed up the calculation.

Notes

Parameter Description Reference: https://github.com/facebook/prophet/blob/main/python/prophet/forecaster.py

property is_prophet_installed

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.ARIMAForecastEstimator(fit_kwargs=None, p=1, d=0, q=0, seasonal_order=(0, 0, 0, 0), trend='c', space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Forecast Estimator based on Hypernets. Estimator: Autoregressive Integrated Moving Average (ARIMA). Suitable for: Univariate Forecast Task.

Parameters:

• p (autoregressive order.) –

• q (moving average order.) –

• d (differences order.) –

• seasonal_order (tuple, optional) – The (P,D,Q,s) order of the seasonal component of the model for the AR parameters, differences, MA parameters, and periodicity. Default is (0, 0, 0, 0). D and s are always integers, while P and Q may either be integers or lists of positive integers.

• trend (str{'n','c','t','ct'} or iterable, optional) – Parameter controlling the deterministic trend. Can be specified as a string where ‘c’ indicates a constant term, ‘t’ indicates a linear trend in time, and ‘ct’ includes both. Can also be specified as an iterable defining a polynomial, as in numpy.poly1d, where [1,1,0,1] would denote \(a + bt + ct^3\). Default is ‘c’ for models without integration, and no trend for models with integration.

Notes

Parameter Description Reference: https://github.com/statsmodels/statsmodels/blob/main/

statsmodels/tsa/arima/model.py

The (p,d,q) order of the model for the autoregressive, differences, and moving average components. d is always an integer, while p and q may either be integers or lists of integers.

• autoregressive models: AR(p)

• moving average models: MA(q)

• mixed autoregressive moving average models: ARMA(p, q)

• integration models: ARIMA(p, d, q)

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.VARForecastEstimator(fit_kwargs=None, maxlags=None, method='ols', ic=None, trend='c', space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Forecast Estimator based on Hypernets. Estimator: Vector Autoregression (VAR). Suitable for: Multivariate Forecast Task.

Parameters:

• maxlags ({int, None}, default None) – Maximum number of lags to check for order selection, defaults to 12 * (nobs/100.)**(1./4), see select_order function

• method ({'ols'}) – Estimation method to use

• ic ({'aic', 'fpe', 'hqic', 'bic', None}) – Information criterion to use for VAR order selection. aic : Akaike fpe : Final prediction error hqic : Hannan-Quinn bic : Bayesian a.k.a. Schwarz

• trend (str {"c", "ct", "ctt", "nc", "n"}) – “c” - add constant “ct” - constant and trend “ctt” - constant, linear and quadratic trend “n”, “nc” - co constant, no trend Note that these are prepended to the columns of the dataset.

Notes

Parameter Description Reference: https://github.com/statsmodels/statsmodels/blob/main/statsmodels/

tsa/vector_ar/var_model.py

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.TSFClassificationEstimator(fit_kwargs=None, min_interval=3, n_estimators=200, n_jobs=1, random_state=None, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Classfication Estimator based on Hypernets. Estimator: Time Series Forest (TSF). Suitable for: Classfication Task.

Parameters:

• n_estimators (int, ensemble size, optional (default = 200)) –

• min_interval (int, minimum width of an interval, optional (default to 3)) –

• n_jobs (int, optional (default=1) The number of jobs to run in parallel for) – both fit and predict. -1 means using all processors.

• random_state (int, seed for random, optional (default = none)) –

Notes

Parameter Description Reference: https://github.com/alan-turing-institute/sktime/blob/main/sktime/

classification/interval_based/_tsf.py

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.KNNClassificationEstimator(fit_kwargs=None, n_neighbors=1, weights='uniform', algorithm='brute', distance='dtw', space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Classfication Estimator based on Hypernets. Estimator: K Nearest Neighbors (KNN). Suitable for: Classfication Task.

Parameters:

• n_neighbors (int, set k for knn (default =1)) –

• weights (string or callable function, optional, default =='uniform') – mechanism for weighting a vote, one of: ‘uniform’, ‘distance’ or a callable function

• algorithm (search method for neighbours {'auto', 'ball_tree',) – ‘kd_tree’, ‘brute’}: default = ‘brute’

• distance (distance measure for time series: {'dtw','ddtw',) – ‘wdtw’,’lcss’,’erp’,’msm’,’twe’}: default =’dtw’

Notes

Parameter Description Reference: https://github.com/alan-turing-institute/sktime/blob/main/sktime/

classification/distance_based/_time_series_neighbors.py

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.IForestDetectionEstimator(fit_kwargs=None, n_estimators=100, max_samples='auto', contamination=0.05, max_features=1.0, bootstrap=False, n_jobs=None, random_state=None, verbose=0, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Anomaly Detection Estimator based on Hypernets. Estimator: Isolation Forest (IForest). Suitable for: Univariate/Multivariate Anomaly Detection Task.

Parameters:

• n_estimators (int, default=100) – The number of base estimators in the ensemble.

• max_samples ("auto", int or float, default="auto") –

  The number of samples to draw from X to train each base estimator.

  • If int, then draw max_samples samples.

  • If float, then draw max_samples * X.shape[0] samples.

  • If “auto”, then max_samples=min(256, n_samples).

  If max_samples is larger than the number of samples provided, all samples will be used for all trees (no sampling).

• contamination ('auto' or float, default=0.05) –

  The amount of contamination of the data set, i.e. the proportion of outliers in the data set. Used when fitting to define the threshold on the scores of the samples.

  • If ‘auto’, the threshold is determined as in the original paper.

  • If float, the contamination should be in the range (0, 0.5].

• max_features (int or float, default=1.0) –

  The number of features to draw from X to train each base estimator.

  • If int, then draw max_features features.

  • If float, then draw max_features * X.shape[1] features.

• bootstrap (bool, default=False) – If True, individual trees are fit on random subsets of the training data sampled with replacement. If False, sampling without replacement is performed.

• n_jobs (int, default=None) – The number of jobs to run in parallel for both fit and predict. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.

• random_state (int, RandomState instance or None, default=None) – Controls the pseudo-randomness of the selection of the feature and split values for each branching step and each tree in the forest. Pass an int for reproducible results across multiple function calls. See Glossary.

• verbose (int, default=0) – Controls the verbosity of the tree building process.

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.OCSVMDetectionEstimator(fit_kwargs=None, kernel='rbf', degree=2, gamma='auto', coef0=0.0, tol=0.001, nu=0.5, shrinking=True, cache_size=200, max_iter=-1, contamination=0.05, verbose=False, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Anomaly Detection Estimator based on Hypernets. Estimator: One Class SVM (OCSVM). Suitable for: Univariate/Multivariate Anomaly Detection Task.

Parameters:

• kernel ({'linear', 'poly', 'rbf', 'sigmoid', 'precomputed'} or callable, default='rbf') – Specifies the kernel type to be used in the algorithm. If none is given, ‘rbf’ will be used. If a callable is given it is used to precompute the kernel matrix.

• degree (int, default=2) – Degree of the polynomial kernel function (‘poly’). Ignored by all other kernels.

• gamma ({'scale', 'auto'} or float, default='scale') –

  Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’. - if gamma='scale' (default) is passed then it uses

  1 / (n_features * X.var()) as value of gamma,

  • if ‘auto’, uses 1 / n_features.

  Changed in version 0.22: The default value of gamma changed from ‘auto’ to ‘scale’.

• coef0 (float, default=0.0) – Independent term in kernel function. It is only significant in ‘poly’ and ‘sigmoid’.

• tol (float, default=1e-3) – Tolerance for stopping criterion.

• nu (float, default=0.5) – An upper bound on the fraction of training errors and a lower bound of the fraction of support vectors. Should be in the interval (0, 1]. By default 0.5 will be taken.

• shrinking (bool, default=True) – Whether to use the shrinking heuristic. See the User Guide.

• cache_size (float, default=200) – Specify the size of the kernel cache (in MB).

• max_iter (int, default=-1) – Hard limit on iterations within solver, or -1 for no limit.

• contamination ('auto' or float, default=0.05) –

  The amount of contamination of the data set, i.e. the proportion of outliers in the data set. Used when fitting to define the threshold on the scores of the samples.

  • If ‘auto’, the threshold is determined as in the original paper.

  • If float, the contamination should be in the range (0, 0.5].

• verbose (bool, default=False) – Enable verbose output. Note that this setting takes advantage of a per-process runtime setting in libsvm that, if enabled, may not work properly in a multithreaded context.

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.DeepARForecastEstimator(fit_kwargs=None, timestamp=None, task='univariate-forecast', rnn_type='gru', rnn_units=16, rnn_layers=1, out_activation='linear', drop_rate=0.0, window=3, horizon=1, forecast_length=1, metrics='auto', monitor='val_loss', optimizer='auto', learning_rate=0.001, loss='log_gaussian_loss', reducelr_patience=5, earlystop_patience=10, summary=True, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, shuffle=True, max_queue_size=10, workers=1, use_multiprocessing=False, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Forecast Estimator based on Hypernets. Estimator: Deep AutoRegressive (DeepAR). Suitable for: Univariate Forecast Task.

Parameters:

• timestamp (Str - Timestamp name, not optional.) –

• task (Str - Only 'univariate-forecast' is supported,) – default = ‘univariate-forecast’.

• rnn_type (Str - Type of recurrent neural network,) – optional {‘basic’, ‘gru’, ‘lstm}, default = ‘gru’.

• rnn_units (Positive Int - The dimensionality of the output space for recurrent neural network,) – default = 16.

• rnn_layers (Positive Int - The number of the layers for recurrent neural network,) – default = 1.

• out_activation (Str - Forecast the task output activation function, optional {'linear', 'sigmoid', 'tanh'},) – default = ‘linear’.

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets,) – default = 0.

• window (Positive Int - Length of the time series sequences for a sample,) – default = 3.

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Only 'log_gaussian_loss' is supported for DeepAR, which has been defined.) – default = ‘log_gaussian_loss’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• batch_size (Int or None - Number of samples per gradient update.) – default = 32.

• epochs (Int - Number of epochs to train the model,) – default = 1.

• verbose (0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment). default = 1.

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. default = None.

• validation_split (Float between 0 and 1.) – Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch, default = 0.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’).

• max_queue_size (Int - Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue, default = 10.

• workers (Int - Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If 0, will execute the generator on the main thread, default = 1.

• use_multiprocessing (Bool. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes. default = False.

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.HybridRNNGeneralEstimator(fit_kwargs=None, timestamp=None, task='univariate-forecast', rnn_type='gru', rnn_units=16, rnn_layers=1, out_activation='linear', drop_rate=0.0, window=3, horizon=1, forecast_length=1, metrics='auto', monitor='val_loss', optimizer='auto', learning_rate=0.001, loss='auto', reducelr_patience=5, earlystop_patience=10, summary=True, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, shuffle=True, max_queue_size=10, workers=1, use_multiprocessing=False, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Forecast|Classification|Regression Estimator based on Hypernets. Estimator: SimpleRNN|GRU|LSTM (HybridRNN). Suitable for: The General Time Series Tasks.

Parameters:

• timestamp (Str or None - Timestamp name, the forecast task must be given.) –

• task (Str - Support forecast, classification, and regression.) – default = ‘univariate-forecast’.

• rnn_type (Str - Type of recurrent neural network,) – optional {‘basic’, ‘gru’, ‘lstm}, default = ‘gru’.

• rnn_units (Positive Int - The dimensionality of the output space for recurrent neural network,) – default = 16.

• rnn_layers (Positive Int - The number of the layers for recurrent neural network,) – default = 1.

• out_activation (Str - Forecast the task output activation function, optional {'linear', 'sigmoid', 'tanh'},) – default = ‘linear’.

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets,) – default = 0.

• window (Positive Int - Length of the time series sequences for a sample,) – default = 3.

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Loss function, optional {'auto', 'adam', 'sgd'},) – default = ‘auto’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• batch_size (Int or None - Number of samples per gradient update.) – default = 32.

• epochs (Int - Number of epochs to train the model,) – default = 1.

• verbose (0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment). default = 1.

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. default = None.

• validation_split (Float between 0 and 1.) – Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch, default = 0.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’).

• max_queue_size (Int - Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue, default = 10.

• workers (Int - Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If 0, will execute the generator on the main thread, default = 1.

• use_multiprocessing (Bool. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes. default = False.

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.LSTNetGeneralEstimator(fit_kwargs=None, timestamp=None, task='univariate-forecast', rnn_type='gru', skip_rnn_type='gru', cnn_filters=16, kernel_size=1, rnn_units=16, rnn_layers=1, skip_rnn_units=16, skip_rnn_layers=1, skip_period=0, ar_order=0, drop_rate=0.0, out_activation='linear', window=7, horizon=1, forecast_length=1, metrics='auto', monitor='val_loss', optimizer='auto', learning_rate=0.001, loss='auto', reducelr_patience=5, earlystop_patience=10, summary=True, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, shuffle=True, max_queue_size=10, workers=1, use_multiprocessing=False, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Forecast|Classification|Regression Estimator based on Hypernets. Estimator: Long-and Short-term Time-series network (LSTNet). Suitable for: The General Time Series Tasks.

Parameters:

• timestamp (Str or None - Timestamp name, the forecast task must be given.) –

• task (Str - Support forecast, classification, and regression.) – default = ‘univariate-forecast’.

• rnn_type (Str - Type of recurrent neural network,) – optional {‘basic’, ‘gru’, ‘lstm}, default = ‘gru’.

• skip_rnn_type (Str - Type of skip recurrent neural network,) – optional {‘simple_rnn’, ‘gru’, ‘lstm}, default = ‘gru’.

• cnn_filters (Positive Int - The dimensionality of the output space (i.e. the number of filters) – in the convolution), default = 16.

• kernel_size (Positive Int - A single integer specifying the spatial dimensions of the filters,) – default = 1.

• rnn_units (Positive Int - The dimensionality of the output space for recurrent neural network,) – default = 16.

• rnn_layers (Positive Int - The number of the layers for recurrent neural network,) – default = 1.

• skip_rnn_units (Positive Int - The dimensionality of the output space for skip recurrent neural network,) – default = 16.

• skip_rnn_layers (Positive Int - The number of the layers for skip recurrent neural network,) – default = 1.

• skip_period (Positive Int or None - The length of skip for recurrent neural network,) – default = None.

• ar_order (Positive Int or None - The window size of the autoregressive component,) – default = None.

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets,) – default = 0.

• out_activation (Str - Forecast the task output activation function, optional {'linear', 'sigmoid', 'tanh'},) – default = ‘linear’.

• window (Positive Int - Length of the time series sequences for a sample,) – default = 7.

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Loss function, optional {'auto', 'adam', 'sgd'},) – default = ‘auto’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• batch_size (Int or None - Number of samples per gradient update,) – default = 32.

• epochs (Int - Number of epochs to train the model,) – default = 1.

• verbose (0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment). default = 1.

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. default = None.

• validation_split (Float between 0 and 1.) – Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch, default = 0.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’).

• max_queue_size (Int - Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue, default = 10.

• workers (Int - Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If 0, will execute the generator on the main thread, default = 1.

• use_multiprocessing (Bool. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes. default = False.

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.NBeatsForecastEstimator(fit_kwargs=None, timestamp=None, task='univariate-forecast', stack_types=('trend', 'seasonality'), thetas_dim=(4, 8), nb_blocks_per_stack=3, share_weights_in_stack=False, hidden_layer_units=256, out_activation='linear', window=3, horizon=1, forecast_length=1, metrics='auto', monitor='val_loss', optimizer='auto', learning_rate=0.001, loss='auto', reducelr_patience=5, earlystop_patience=10, summary=True, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, shuffle=True, max_queue_size=10, workers=1, use_multiprocessing=False, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Forecast Estimator based on Hypernets. Estimator: Neural Basis Expansion Analysis For Interpretable Time Series Forecasting (NBeats). Suitable for: Univariate/Multivariate Forecast Task.

Parameters:

• timestamp (Str - Timestamp name, not optional.) –

• task (Str - Only 'forecast' is supported,) – default = ‘univariate-forecast’.

• stack_types (Tuple(Str) - Stack types, optional {'trend', 'seasonality', generic}.) – default = (‘trend’, ‘seasonality’).

• thetas_dim (Tuple(Int) - The number of units that make up each dense layer in each block of every stack.) – default = (4, 8).

• nb_blocks_per_stack (Int - The number of block per stack.) – default = 3.

• share_weights_in_stack (Bool - Whether to share weights in stack.) – default = False.

• hidden_layer_units (Int - The units of hidden layer.) – default = 256.

• out_activation (Str - Forecast the task output activation function, optional {'linear', 'sigmoid', 'tanh'},) – default = ‘linear’.

• window (Positive Int - Length of the time series sequences for a sample,) – default = 3.

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Only 'log_gaussian_loss' is supported for DeepAR, which has been defined.) – default = ‘log_gaussian_loss’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• batch_size (Int or None - Number of samples per gradient update.) – default = 32.

• epochs (Int - Number of epochs to train the model,) – default = 1.

• verbose (0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment). default = 1.

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. default = None.

• validation_split (Float between 0 and 1.) – Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch, default = 0.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’).

• max_queue_size (Int - Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue, default = 10.

• workers (Int - Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If 0, will execute the generator on the main thread, default = 1.

• use_multiprocessing (Bool. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes. default = False.

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.InceptionTimeGeneralEstimator(fit_kwargs=None, timestamp=None, task='univariate-binaryclass', blocks=3, cnn_filters=32, kernel_size_list=(1, 3, 5, 8, 12), bottleneck_size=32, shortcut=True, short_filters=64, metrics='auto', monitor='val_loss', optimizer='auto', learning_rate=0.001, loss='auto', reducelr_patience=5, earlystop_patience=10, summary=True, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, shuffle=True, max_queue_size=10, workers=1, use_multiprocessing=False, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Classification or Regression Estimator based on Hypernets. Estimator: Inception Time (InceptionTime). Suitable for: Univariate/Multivariate Classification or Regression Task.

Parameters:

• timestamp (Str - Timestamp name, not optional.) –

• task (Str - Only 'classification' is supported,) – default = ‘univariate-binaryclass’.

• blocks (Int - The depth of the net architecture.) – default = 3.

• cnn_filters (Int - The number of cnn filters.) – default = 32.

• bottleneck_size (Int - The number of bottleneck (a cnn layer).) – default = 32.

• kernel_size_list (Tuple - The kernel size of cnn for a inceptionblock.) – default = (1, 3, 5, 8, 12).

• shortcut (Bool - Whether to use shortcut opration.) – default = True.

• short_filters (Int - The number of filters of shortcut conv1d layer.) – default = 64.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Only 'log_gaussian_loss' is supported for DeepAR, which has been defined.) – default = ‘log_gaussian_loss’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• batch_size (Int or None - Number of samples per gradient update.) – default = 32.

• epochs (Int - Number of epochs to train the model,) – default = 1.

• verbose (0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment). default = 1.

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. default = None.

• validation_split (Float between 0 and 1.) – Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch, default = 0.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’).

• max_queue_size (Int - Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue, default = 10.

• workers (Int - Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If 0, will execute the generator on the main thread, default = 1.

• use_multiprocessing (Bool. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes. default = False.

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.estimators.ConvVAEDetectionEstimator(fit_kwargs=None, timestamp=None, task='detection', contamination=0.05, window=3, horizon=1, forecast_length=1, latent_dim=2, conv_type='general', cnn_filters=16, kernel_size=1, strides=1, nb_layers=2, activation='relu', drop_rate=0.2, out_activation='linear', reconstract_dim=None, metrics='auto', monitor_metric='val_loss', optimizer='auto', learning_rate=0.001, reducelr_patience=5, earlystop_patience=10, summary=True, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, shuffle=True, max_queue_size=10, workers=1, use_multiprocessing=False, space=None, name=None, **kwargs)

Bases: HyperEstimator

Time Series Anomaly Detection Estimator based on Hypernets. Estimator: Convolution Variational AutoEncoder (ConvVAE). Suitable for: Univariate/Multivariate Anomaly Detection Task.

Parameters:

• task (Str - Only support anomaly detection.) – See hyperts.utils.consts for details.

• timestamp (Str or None - Timestamp name, the forecast task must be given,) – default None.

• window (Positive Int - Length of the time series sequences for a sample.) –

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• latent_dim (Int - Latent representation of encoder, default 2.) –

• conv_type (Str - Type of 1D convolution, optional {'general', 'separable'},) – default ‘general’.

• cnn_filters (Positive Int - The dimensionality of the output space (i.e. the number) – of filters in the convolution).

• kernel_size (Positive Int - A single integer specifying the spatial dimensions) – of the filters,

• strides (Int or tuple/list of a single integer - Specifying the stride length) – of the convolution.

• nb_layers (Int - The layers of encoder and decoder, default 2.) –

• activation (Str - The activation of hidden layers, default 'relu'.) –

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets.) –

• out_activation (Str - Forecast the task output activation function,) – optional {‘linear’, ‘sigmoid’, ‘tanh’}, default ‘linear’.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor_metric (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• batch_size (Int or None - Number of samples per gradient update.) – default = 32.

• epochs (Int - Number of epochs to train the model,) – default = 1.

• verbose (0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment). default = 1.

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. default = None.

• validation_split (Float between 0 and 1.) – Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch, default = 0.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’).

• max_queue_size (Int - Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue, default = 10.

• workers (Int - Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If 0, will execute the generator on the main thread, default = 1.

• use_multiprocessing (Bool. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes. default = False.

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

build_estimator(task=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

framework.wrappers

wrappers.dl_wrappers

class hyperts.framework.wrappers.dl_wrappers.DeepARWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: univariate forecast.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

predict_proba(X, **kwargs)

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.dl_wrappers.HybridRNNWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: forecast, classification and regression.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

predict_proba(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.dl_wrappers.LSTNetWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: forecast, classification and regression.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

predict_proba(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.dl_wrappers.NBeatsWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: forecast.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

predict_proba(X, **kwargs)

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.dl_wrappers.InceptionTimeWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: classification.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

predict_proba(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.dl_wrappers.ConvVAEWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: anomaly detection.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

predict_proba(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

wrappers.nas_wrappers

class hyperts.framework.wrappers.nas_wrappers.TSNASWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: forecast, classification and regression.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

predict_proba(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

wrappers.stats_wrappers

class hyperts.framework.wrappers.stats_wrappers.ProphetWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: univariate forecast.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

predict_proba(X, **kwargs)

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.stats_wrappers.ARIMAWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: univariate forecast.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

predict_proba(X, **kwargs)

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.stats_wrappers.VARWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: multivariate forecast.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

predict_proba(X, **kwargs)

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.stats_wrappers.TSForestWrapper(fit_kwargs=None, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: univariate classification.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

predict_proba(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.stats_wrappers.KNeighborsWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: univariate/multivariate classification.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

predict_proba(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.stats_wrappers.IForestWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: univariate/multivariate anomaly detection.

fit(X, y=None, **kwargs)

predict(X, **kwargs)

predict_proba(X, **kwargs)

property classes_

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

class hyperts.framework.wrappers.stats_wrappers.OneClassSVMWrapper(fit_kwargs, **kwargs)

Bases: EstimatorWrapper, WrapperMixin

Adapt: univariate/multivariate anomaly detection.

detection_split_XTC(XTC)

drop_hist_sample(X, y=None, **kwargs)

fit(X, y=None, **kwargs)

fit_transform(X)

inverse_transform(X)

property logx

property outlier

property scaler

transform(X)

update_fit_kwargs()

update_init_kwargs(**kwargs)

predict(X, **kwargs)

predict_proba(X, **kwargs)

property classes_

framework.search_space

search_space.macro_search_space

hyperts.framework.search_space.macro_search_space.categorical_transform_pipeline(covariables=None, impute_strategy=None, seq_no=0)

hyperts.framework.search_space.macro_search_space.numeric_transform_pipeline(covariables=None, impute_strategy=None, seq_no=0)

class hyperts.framework.search_space.macro_search_space.BaseSearchSpaceGenerator(task, **kwargs)

Bases: object

property estimators

create_preprocessor(hyper_input, options)

create_estimators(hyper_input, options)

class hyperts.framework.search_space.macro_search_space.StatsForecastSearchSpace(task=None, timestamp=None, enable_prophet=True, enable_arima=True, enable_var=True, prophet_init_kwargs=None, arima_init_kwargs=None, var_init_kwargs=None, drop_observed_sample=True, **kwargs)

Bases: BaseSearchSpaceGenerator, SearchSpaceMixin

Statistical Search Space for Time Series Forecasting.

Parameters:

• task (str or None, optional, default None. If not None, it must be 'univariate-forecast' or) – ‘multivariate-forecast’.

• timestamp (str or None, optional, default None.) –

• metrics (str or None, optional, default None. Support mse, mae, rmse, mape, smape, msle, and so on.) –

• enable_prophet (bool, default True.) –

• enable_arima (bool, default True.) –

• enable_var (bool, default True.) –

• prophet_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which prophet is searched.

• arima_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which arima is searched.

• var_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which var is searched.

Return type:

search space.

Notes

1. For the hyper-parameters of deepar_init_kwargs, hybirdrnn_init_kwargs and lstnet_init_kwargs,

   you can refer to hyperts.framework.estimators.ProphetForecastEstimator, hyperts.framework.estimators.ARIMAForecastEstimator, and hyperts.framework.estimators.VARForecastEstimator.

2. If other parameters exist, set them directly. For example, covariables=[‘is_holiday’].

property default_prophet_init_kwargs

property default_prophet_fit_kwargs

property default_arima_init_kwargs

property default_arima_fit_kwargs

property default_var_init_kwargs

property default_var_fit_kwargs

property estimators

create_estimators(hyper_input, options)

create_preprocessor(hyper_input, options)

initial_window_kwargs(default_init_kwargs)

update_init_params(**kwargs)

class hyperts.framework.search_space.macro_search_space.StatsClassificationSearchSpace(task=None, timestamp=None, enable_tsf=True, enable_knn=True, tsf_init_kwargs=None, knn_init_kwargs=None, **kwargs)

Bases: BaseSearchSpaceGenerator, SearchSpaceMixin

Statistical Search Space for Time Series Classification.

Parameters:

• task (str or None, optional, default None. If not None, it must be 'univariate-binaryclass',) – ‘univariate-multiclass’, ‘multivariate-binaryclass, or ’multivariate-multiclass’.

• timestamp (str or None, optional, default None.) –

• metrics (str or None, optional, default None. Support accuracy, f1, auc, recall, precision.) –

• enable_tsf (bool, default True.) –

• enable_knn (bool, default True.) –

• tsf_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which tsf is searched.

• knn_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which knn is searched.

Return type:

search space.

Notes

1. For the hyper-parameters of tsf_init_kwargs, knn_init_kwargs,

   you can refer to hyperts.framework.estimators.TSFClassificationEstimator, and hyperts.framework.estimators.KNNClassificationEstimator.

2. If other parameters exist, set them directly. For example, n_estimators=200.

property default_tsf_init_kwargs

property default_tsf_fit_kwargs

property default_knn_univar_init_kwargs

property default_knn_multivar_init_kwargs

property default_knn_fit_kwargs

property estimators

create_estimators(hyper_input, options)

create_preprocessor(hyper_input, options)

initial_window_kwargs(default_init_kwargs)

update_init_params(**kwargs)

class hyperts.framework.search_space.macro_search_space.DLForecastSearchSpace(task=None, timestamp=None, metrics=None, window=None, horizon=1, enable_deepar=True, enable_hybridrnn=True, enable_lstnet=True, enable_nbeats=True, deepar_init_kwargs=None, hybridrnn_init_kwargs=None, lstnet_init_kwargs=None, nbeats_init_kwargs=None, drop_observed_sample=True, **kwargs)

Bases: BaseSearchSpaceGenerator, SearchSpaceMixin

Deep Learning Search Space for Time Series Forecasting.

Parameters:

• task (str or None, optional, default None. If not None, it must be 'univariate-forecast' or) – ‘multivariate-forecast’.

• timestamp (str or None, optional, default None.) –

• metrics (str or None, optional, default None. Support mse, mae, rmse, mape, smape, msle, and so on.) –

• enable_deepar (bool, default True.) –

• enable_hybridrnn (bool, default True.) –

• enable_lstnet (bool, default True.) –

• deepar_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which deepar is searched.

• hybirdrnn_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which hybirdrnn is searched.

• lstnet_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which lstnet is searched.

Return type:

search space.

Notes

1. For the hyper-parameters of deepar_init_kwargs, hybridrnn_init_kwargs and lstnet_init_kwargs,

   you can refer to hyperts.framework.estimators.DeepARForecastEstimator, hyperts.framework.estimators.HybridRNNGeneralEstimator, and hyperts.framework.estimators.LSTNetGeneralEstimator.

2. If other parameters exist, set them directly. For example, covariables=[‘is_holiday’].

property default_deepar_init_kwargs

property default_deepar_fit_kwargs

property default_hybridrnn_init_kwargs

property default_hybridrnn_fit_kwargs

property default_lstnet_init_kwargs

property default_lstnet_fit_kwargs

property default_nbeats_init_kwargs

property default_nbeats_fit_kwargs

property estimators

create_estimators(hyper_input, options)

create_preprocessor(hyper_input, options)

initial_window_kwargs(default_init_kwargs)

update_init_params(**kwargs)

class hyperts.framework.search_space.macro_search_space.DLClassRegressSearchSpace(task=None, timestamp=None, metrics=None, enable_hybridrnn=True, enable_lstnet=True, enable_inceptiontime=True, hybridrnn_init_kwargs=None, lstnet_init_kwargs=None, inceptiontime_init_kwargs=None, **kwargs)

Bases: BaseSearchSpaceGenerator, SearchSpaceMixin

Deep Learning Search Space for Time Series Classification and Regression.

Parameters:

• task (str or None, optional, default None. If not None, it must be 'univariate-binaryclass',) – ‘univariate-multiclass’, ‘multivariate-binaryclass, or ’multivariate-multiclass’.

• timestamp (str or None, optional, default None.) –

• metrics (str or None, optional, default None. Support accuracy, f1, auc, recall, precision.) –

• enable_hybirdrnn (bool, default True.) –

• enable_lstnet (bool, default True.) –

• hybridrnn_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which hybirdrnn is searched.

• lstnet_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which lstnet is searched.

Return type:

search space.

Notes

1. For the hyper-parameters of deepar_init_kwargs, hybridrnn_init_kwargs and lstnet_init_kwargs,

   you can refer to hyperts.framework.estimators.HybirdRNNGeneralEstimator, and hyperts.framework.estimators.LSTNetGeneralEstimator.

2. If other parameters exist, set them directly. For example, n_estimators=200.

property default_hybridrnn_init_kwargs

property default_hybridrnn_fit_kwargs

property default_lstnet_init_kwargs

property default_lstnet_fit_kwargs

property default_inceptiontime_init_kwargs

property default_inceptiontime_fit_kwargs

property estimators

create_estimators(hyper_input, options)

create_preprocessor(hyper_input, options)

initial_window_kwargs(default_init_kwargs)

update_init_params(**kwargs)

class hyperts.framework.search_space.macro_search_space.StatsDetectionSearchSpace(task=None, timestamp=None, enable_iforest=True, enable_ocsvm=True, iforest_init_kwargs=None, ocsvm_init_kwargs=None, drop_observed_sample=False, **kwargs)

Bases: BaseSearchSpaceGenerator, SearchSpaceMixin

Statistical Search Space for Time Series Anomaly Detection.

Parameters:

• task (str or None, optional, default None. If not None, it must be 'detection'.) –

• timestamp (str or None, optional, default None.) –

• metrics (str or None, optional, default None. Support f1, precision, recall, and so on.) –

• enable_iforest (bool, default True.) –

• enable_ocsvm (bool, default True.) –

• iforest_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which prophet is searched.

• ocsvm_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which prophet is searched.

Return type:

search space.

Notes

1. For the hyper-parameters of iforest_init_kwargs, ocsvm_init_kwargs,

   you can refer to hyperts.framework.estimators.IforestDetectionEstimator, ‘hyperts.framework.estimators.OCSVMDetectionEstimator’.

2. If other parameters exist, set them directly. For example, covariables=[‘is_holiday’].

property default_iforest_init_kwargs

property default_iforest_fit_kwargs

property default_ocsvm_init_kwargs

property default_ocsvm_fit_kwargs

property estimators

create_estimators(hyper_input, options)

create_preprocessor(hyper_input, options)

initial_window_kwargs(default_init_kwargs)

update_init_params(**kwargs)

class hyperts.framework.search_space.macro_search_space.DLDetectionSearchSpace(task=None, timestamp=None, metrics=None, window=None, horizon=1, enable_conv_vae=True, conv_vae_init_kwargs=None, drop_observed_sample=False, **kwargs)

Bases: BaseSearchSpaceGenerator, SearchSpaceMixin

Deep Learning Search Space for Time Series Anomaly Detection.

Parameters:

• task (str or None, optional, default None. If not None, it must be 'detection'.) –

• timestamp (str or None, optional, default None.) –

• metrics (str or None, optional, default None. Support f1, precision, recall, and so on.) –

• enable_convvae (bool, default True.) –

• convvae_init_kwargs (dict or None, optional, default None. If not None, you can customize) – the hyper-parameters by which prophet is searched.

Return type:

search space.

Notes

1. For the hyper-parameters of iforest_init_kwargs, ocsvm_init_kwargs,

   you can refer to hyperts.framework.estimators.ConvVAEDetectionEstimator.

2. If other parameters exist, set them directly. For example, covariables=[‘is_holiday’].

property default_conv_vae_init_kwargs

property default_conv_vae_fit_kwargs

property estimators

create_estimators(hyper_input, options)

create_preprocessor(hyper_input, options)

initial_window_kwargs(default_init_kwargs)

update_init_params(**kwargs)

search_space.micro_search_space

class hyperts.framework.search_space.micro_search_space.TSNASGenrealSearchSpace(num_blocks=2, num_nodes=4, init_filters_or_units=64, block_ops='concat', name='tsnas', drop_observed_sample=True, **kwargs)

Bases: SearchSpaceMixin

TS-NAS Search Space for Time Series Forecasting, Classification and Regression.

Parameters:

• num_blocks (int, the number of blocks.) –

• num_nodes (int, the number of nodes.) –

• init_filters_or_units (int, the number of filters(CNN) or units(RNN).) –

• block_ops (str, the operations for node of block, optional {'add', 'concat'}.) –

Return type:

search space.

property default_forecasting_init_kwargs

property default_classification_regression_init_kwargs

initial_window_kwargs(default_init_kwargs)

update_init_params(**kwargs)

hyperts.framework.search_space.micro_search_space.search_space_univariate_forecast_generator(covariate=(), timestamp=None)

hyperts.framework.search_space.micro_search_space.search_space_multivariate_forecast_generator(covariate=(), timestamp=None)

hyperts.framework.search_space.micro_search_space.search_space_multivariate_classification()

framework.meta_learning

meta_learning.helper_fn

hyperts.framework.meta_learning.helper_fn.ptp(X)

Range of values (maximum - minimum) along an axis.

hyperts.framework.meta_learning.helper_fn.normalization(metafeatures)

Normalized meta features.

hyperts.framework.meta_learning.helper_fn.warm_start_sample(space_sample, trial_store)

Initialize the HyperSpace with promising meta-hyperparameter.

Parameters:

• space_sample (HyperSpace class.) –

• trial_store (TrialStore class.) –

meta_learning.meta_learner

class hyperts.framework.meta_learning.meta_learner.MetaLearner(history, dataset_id, trial_store, **kwargs)

Bases: MetaLearner

fit(space_signature)

extract_features_and_labels(signature)

new_sample(space_sample)

predict(space_sample, default_value=inf)

meta_learning.meta_trial_store

class hyperts.framework.meta_learning.meta_trial_store.TrialInstance(signature, vectors, reward, elapsed, run=False)

Bases: object

Trial Instance.

Parameters:

• signature (str, trial instance signature.) –

• vectors (list, configuration vectors of space_sample.) –

• reward (float, reward of space_sample.) –

• elapsed (float, trial elapsed.) –

• run (bool, whether the current data has been run (fit), default False.) –

class hyperts.framework.meta_learning.meta_trial_store.TrialStore(task, dataset_id, reward_metric, is_scale=True, trials_limit=30, **kwargs)

Bases: object

Trial store for meta learning to warm start optimization.

Parameters:

• task (str, task name, for example, 'univariate-forecast', 'multivariate-forecast' and so on.) –

• dataset_id (str, dataset id based on shape and dtype (X, y).) –

• reward_metric ('str' or callable, default 'accuracy' for binary/multiclass task, 'rmse' for) – forecast/regression task.

• is_scale (bool, whether to scale metafeatures, default True.) –

• trials_limit (int, number of collection trials, default 30.) –

fit(X, y=None)

Calculate metafeatures and configuration information.

get(dataset_id, space_sample)

Extract meta trial from trial store.

put(dataset_id, new_trial)

Put trial into trials.

get_all(dataset_id, space_signature)

Extract all trial configurations that satisfy the specified space_sample.

get_metafeatures(features=None)

Extract meta-features.

get_configurations(reward_metric)

Extract the trial configurations of history.

framework.meta_learning.tsfeatures

hyperts.framework.meta_learning.tsfeatures.scale(x)

Z-Score.

Parameters:

x (np.array or pd.DataFrame, the time series.) –

hyperts.framework.meta_learning.tsfeatures.fft_infer_period(x)

Fourier inference period.

Parameters:

x (np.array or pd.DataFrame, the time series.) –

References

1. https://github.com/xuawai/AutoPeriod/blob/master/auto_period.ipynb

hyperts.framework.meta_learning.tsfeatures.freq_to_numerical(x, timestamp, freq_mapping_dict=None)

Calculates frequency.

Parameters:

• x (pd.DataFrame, the timestamp.) –

• timestamp (str, timestamp name of x.) –

• freq_mapping_dict (24, 'D': 7, 'W': 54, 'M': 12,) – ‘Q’: 4, ‘Y’: 1, ‘A’: 1, ‘S’: 60, ‘T’: 60}.

• dict (24, 'D': 7, 'W': 54, 'M': 12,) – ‘Q’: 4, ‘Y’: 1, ‘A’: 1, ‘S’: 60, ‘T’: 60}.

• {'H' (default) – ‘Q’: 4, ‘Y’: 1, ‘A’: 1, ‘S’: 60, ‘T’: 60}.

hyperts.framework.meta_learning.tsfeatures.statistics(x, period: int = 1)

Calculates statistics features, including length, count, mean, var, min, max, median, range, hmean, iqr.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

hyperts.framework.meta_learning.tsfeatures.acf_features(x, period: int = 1)

Calculates autocorrelation function features.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

hyperts.framework.meta_learning.tsfeatures.pacf_features(x, period: int = 1)

Calculates partial autocorrelation function features.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

hyperts.framework.meta_learning.tsfeatures.crossing_points(x, period: int = 1)

Crossing points.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

hyperts.framework.meta_learning.tsfeatures.stability(x, period: int = 10)

Calculate the stability of time series.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

hyperts.framework.meta_learning.tsfeatures.lumpiness(x, period: int = 10)

Calculating the lumpiness of time series.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

hyperts.framework.meta_learning.tsfeatures.entropy(x, period: int = 1)

Calculates spectral entropy.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

hyperts.framework.meta_learning.tsfeatures.hurst(x, period: int = 30)

Calculates hurst exponet.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

hyperts.framework.meta_learning.tsfeatures.stl_features(x, period: int = 1)

Calculates the strength of trend and seasonality, spikiness, linearity, curvature, peak, trough, e_acf1.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

3. https://github.com/Nixtla/tsfeatures/blob/master/tsfeatures (Python code)

hyperts.framework.meta_learning.tsfeatures.holt_parameters(x, period: int = 1)

Calculates the parameters of a Holt model.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

hyperts.framework.meta_learning.tsfeatures.hw_parameters(x: ndarray, period: int = 1)

Calculates the parameters of a Holt-Winters model.

Parameters:

• x (np.array or pd.DataFrame, the time series.) –

• period (int, the seasonal of the time series.) –

References

1. T.S. Talagala, et al., Meta-learning how to forecast time series, 2018.

2. https://github.com/robjhyndman/tsfeatures (R code)

hyperts.framework.meta_learning.tsfeatures.metafeatures_from_timeseries(x, timestamp, period=None, scale_ts=True, freq_mapping_dict=None, features_list=None)

Extracting the meta-features of time series.

Parameters:

• x (pd.DataFrame, the time series.) –

• timestamp (str, timestamp name of x.) –

• period (int or None, the seasonal of the time series, default None.) –

• scale_ts (bool, whether scale original time series.) –

• freq_mapping_dict (24, 'D': 7, 'W': 54, 'M': 12,) – ‘Q’: 4, ‘Y’: 1, ‘A’: 1, ‘S’: 60, ‘T’: 60}.

• dict (24, 'D': 7, 'W': 54, 'M': 12,) – ‘Q’: 4, ‘Y’: 1, ‘A’: 1, ‘S’: 60, ‘T’: 60}.

• {'H' (default) – ‘Q’: 4, ‘Y’: 1, ‘A’: 1, ‘S’: 60, ‘T’: 60}.

• features_list –

• List[str] –

• ['simple' (default) –

• 'all']. –

framework.stats

class hyperts.framework.stats.iforest.TSIsolationForest(n_estimators=100, max_samples='auto', contamination=0.05, max_features=1.0, bootstrap=False, n_jobs=None, random_state=None, verbose=0, name='isolation_forest')

Bases: BaseAnomalyDetectorWrapper

Isolation Forest for anomaly detection.

Parameters:

• n_estimators (int, default=100) – The number of base estimators in the ensemble.

• max_samples ("auto", int or float, default="auto") –

  The number of samples to draw from X to train each base estimator.

  • If int, then draw max_samples samples.

  • If float, then draw max_samples * X.shape[0] samples.

  • If “auto”, then max_samples=min(256, n_samples).

  If max_samples is larger than the number of samples provided, all samples will be used for all trees (no sampling).

• contamination (float, default=0.05) –

  The amount of contamination of the data set, i.e. the proportion of outliers in the data set. Used when fitting to define the threshold on the scores of the samples.

  • If ‘auto’, the threshold is determined as in the original paper.

  • If float, the contamination should be in the range (0, 0.5].

• max_features (int or float, default=1.0) –

  The number of features to draw from X to train each base estimator.

  • If int, then draw max_features features.

  • If float, then draw max_features * X.shape[1] features.

• bootstrap (bool, default=False) – If True, individual trees are fit on random subsets of the training data sampled with replacement. If False, sampling without replacement is performed.

• n_jobs (int, default=None) – The number of jobs to run in parallel for both fit and predict. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.

• random_state (int, RandomState instance or None, default=None) –

  Controls the pseudo-randomness of the selection of the feature and split values for each branching step and each tree in the forest.

  Pass an int for reproducible results across multiple function calls. See Glossary.

• verbose (int, default=0) – Controls the verbosity of the tree building process.

decision_function(X)

Predict anomaly scores for sequences in X.

Parameters:

X (numpy array of shape (n_samples, n_features).) –

Returns:

anomaly_scores – The anomaly score of the input samples.

Return type:

numpy array of shape (n_samples,)

fit(X, y=None, **kwargs)

Fit time series model to training data.

Parameters:

• X (numpy array og shape (n_samples, n_features).) –

• y (ignored in unsupervised methods. default None.) –

Returns:

self

Return type:

object.

predict(X, **kwargs)

Predict labels for sequences in X.

Parameters:

X (numpy array of shape (n_samples, n_features).) –

Returns:

outlier_labels – For each observation, tells whether or not it should be considered as an outlier according to the fitted model. 0 stands for inliers and 1 for outliers.

Return type:

numpy array of shape (n_samples,)

predict_confidence(X)

Predict the confidence of model in making the same prediction

under slightly different training sets.

Parameters:

X (numpy array of shape (n_samples, n_features).) –

Returns:

• confidence (numpy array of shape (n_samples,).) – For each observation, tells how consistently the model would make the same prediction if the training set was perturbed. Return a probability, ranging in [0,1].

• Reference

• ———

• https (//github.com/yzhao062/pyod)

predict_proba(X, methed='erf')

Predict the probability for sequences in X.

Parameters:

• X (numpy array of shape (n_samples, n_features).) –

• methed (str, optional {'erf', 'linear'}. Probability conversion method.) –

Returns:

outlier_probability – For each observation, tells whether or not it should be considered as an outlier according to the fitted model. Return the outlier probability, ranging in [0,1]. Note it depends on the number of classes, which is by default 2 classes ([proba of normal, proba of outliers]).

Return type:

numpy array of shape (n_samples, n_classes)

class hyperts.framework.stats.ocsvm.TSOneClassSVM(kernel='rbf', degree=2, gamma='auto', coef0=0.0, tol=0.001, nu=0.5, shrinking=True, cache_size=200, max_iter=-1, contamination=0.05, verbose=False, name='one class svm')

Bases: BaseAnomalyDetectorWrapper

One-Class Support Vector Mechine for anomaly detection.

Parameters:

• kernel ({'linear', 'poly', 'rbf', 'sigmoid', 'precomputed'} or callable, default='rbf') – Specifies the kernel type to be used in the algorithm. If none is given, ‘rbf’ will be used. If a callable is given it is used to precompute the kernel matrix.

• degree (int, default=2) – Degree of the polynomial kernel function (‘poly’). Ignored by all other kernels.

• gamma ({'scale', 'auto'} or float, default='scale') –

  Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’.

  • if gamma='scale' (default) is passed then it uses 1 / (n_features * X.var()) as value of gamma,

  • if ‘auto’, uses 1 / n_features.

  Changed in version 0.22: The default value of gamma changed from ‘auto’ to ‘scale’.

• coef0 (float, default=0.0) – Independent term in kernel function. It is only significant in ‘poly’ and ‘sigmoid’.

• tol (float, default=1e-3) – Tolerance for stopping criterion.

• nu (float, default=0.5) – An upper bound on the fraction of training errors and a lower bound of the fraction of support vectors. Should be in the interval (0, 1]. By default 0.5 will be taken.

• shrinking (bool, default=True) – Whether to use the shrinking heuristic. See the User Guide.

• cache_size (float, default=200) – Specify the size of the kernel cache (in MB).

• max_iter (int, default=-1) – Hard limit on iterations within solver, or -1 for no limit.

• contamination ('auto' or float, default=0.05) –

  The amount of contamination of the data set, i.e. the proportion of outliers in the data set. Used when fitting to define the threshold on the scores of the samples.

  • If ‘auto’, the threshold is determined as in the original paper.

  • If float, the contamination should be in the range (0, 0.5].

• verbose (bool, default=False) – Enable verbose output. Note that this setting takes advantage of a per-process runtime setting in libsvm that, if enabled, may not work properly in a multithreaded context.

decision_function(X)

Predict anomaly scores for sequences in X.

Parameters:

X (numpy array of shape (n_samples, n_features).) –

Returns:

anomaly_scores – The anomaly score of the input samples.

Return type:

numpy array of shape (n_samples,)

fit(X, y=None, **kwargs)

Fit time series model to training data.

Parameters:

• X (numpy array og shape (n_samples, n_features).) –

• y (ignored in unsupervised methods. default None.) –

Returns:

self

Return type:

object.

predict(X, **kwargs)

Predict labels for sequences in X.

Parameters:

X (numpy array of shape (n_samples, n_features).) –

Returns:

outlier_labels – For each observation, tells whether or not it should be considered as an outlier according to the fitted model. 0 stands for inliers and 1 for outliers.

Return type:

numpy array of shape (n_samples,)

predict_confidence(X)

Predict the confidence of model in making the same prediction

under slightly different training sets.

Parameters:

X (numpy array of shape (n_samples, n_features).) –

Returns:

• confidence (numpy array of shape (n_samples,).) – For each observation, tells how consistently the model would make the same prediction if the training set was perturbed. Return a probability, ranging in [0,1].

• Reference

• ———

• https (//github.com/yzhao062/pyod)

predict_proba(X, methed='erf')

Predict the probability for sequences in X.

Parameters:

• X (numpy array of shape (n_samples, n_features).) –

• methed (str, optional {'erf', 'linear'}. Probability conversion method.) –

Returns:

outlier_probability – For each observation, tells whether or not it should be considered as an outlier according to the fitted model. Return the outlier probability, ranging in [0,1]. Note it depends on the number of classes, which is by default 2 classes ([proba of normal, proba of outliers]).

Return type:

numpy array of shape (n_samples, n_classes)

framework.dl

dl.base

class hyperts.framework.dl._base.Metrics(*args, **kwargs)

Bases: UserDict

A User Dict class to store metrics required for model configuration.

Usage with compile() API.

Return type:

A tf.keras.metrics.Metirc object.

clear() → None.  Remove all items from D.

copy()

classmethod fromkeys(iterable, value=None)

get(k[, d]) → D[k] if k in D, else d.  d defaults to None.

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D

update([E, ]**F) → None.  Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() → an object providing a view on D's values

class hyperts.framework.dl._base.Losses(*args, **kwargs)

Bases: UserDict

A User Dict class to store loss required for model configuration.

Usage with compile() API.

Return type:

A tf.keras.metrics.Loss object.

clear() → None.  Remove all items from D.

copy()

classmethod fromkeys(iterable, value=None)

get(k[, d]) → D[k] if k in D, else d.  d defaults to None.

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D

update([E, ]**F) → None.  Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() → an object providing a view on D's values

class hyperts.framework.dl._base.BaseDeepEstimator(task, timestamp=None, window=None, horizon=None, forecast_length=1, monitor_metric=None, reducelr_patience=None, earlystop_patience=None, embedding_output_dim=4, continuous_columns=None, categorical_columns=None)

Bases: object

Abstract base class representing deep estimator object.

Parameters:

• task ('str' or None, default None.) – Task could be ‘univariate-forecast’, ‘multivariate-forecast’, and ‘univariate-binaryclass’, etc. See consts.py for details.

• timestamp ('str' or None, default None, representing time column name (in DataFrame).) –

• window ('int' or None, default None, length of the time series sequences for a sample.) – This must be specified for a forecast task.

• horizon ('int' or None, default None, representing the time interval between the start point) – of prediction time and the end point of observation time. This must be specified for a forecast task.

• forecast_length ('int', default 1.) – A forecast field of vision during a forecast task.

• monitor_metric ('str' or None, default None.) – Quantity to be monitored.

• reducelr_patience ('int' or None, default None.) – Number of epochs with no improvement after which learning rate will be reduced.

• earlystop_patience ('str' or None, default None.) – Number of epochs with no improvement after which training will be stopped.

• embedding_output_dim ('int', default 4.) – The dimension in which categorical variables are embedded.

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

fit(X, y, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.2, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1, max_queue_size=10, workers=1, use_multiprocessing=False, **kwargs)

Trains the model for a fixed number of epochs (iterations on a dataset).

Parameters:

• X (2D DataFrame of shape: (series_length, 1+(n_covariates)) for forecast task,) – 2D nested DataFrame shape: (sample, nb_covariables(series_length)) for classification or regression task, 2D DataFrame of shape (n_samples, 1+(n_covariates)) for anomaly detection.

• y (2D DataFrame of shape: (series_length, n_target_variables) for forecast task,) – 2D DataFrame of shape: (nb_samples, 1) for classification or regression task, 2D DataFrame of shape (n_samples, reconstract_dim).

• batch_size (Integer or None.) – Number of samples per gradient update. If unspecified, batch_size will default to self-adaption based on number of samples. Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

• epochs (Integer. Number of epochs to train the model.) – An epoch is an iteration over the entire x and y data provided. Note that in conjunction with initial_epoch, epochs is to be understood as “final epoch”. The model is not trained for a number of iterations given by epochs, but merely until the epoch of index epochs is reached.

• verbose ('auto', 0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. ‘auto’ defaults to 1 for most cases, but 2 when used with ParameterServerStrategy. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment).

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. Callbacks with batch-level calls are currently unsupported with tf.distribute.experimental.ParameterServerStrategy, and users are advised to implement epoch-level calls instead with an appropriate steps_per_epoch value.

• validation_split (Float between 0 and 1.) –

  Fraction of the training data to be used as validation data.

  The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling. This argument is not supported when x is a dataset, generator or

  keras.utils.Sequence instance.

  validation_split is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• validation_data (Data on which to evaluate) –

  the loss and any model metrics at the end of each epoch. The model will not be trained on this data. Thus, note the fact that the validation loss of data provided using validation_split or validation_data is not affected by regularization layers like noise and dropout. validation_data will override validation_split. validation_data could be:

  • A tuple (x_val, y_val) of Numpy arrays or tensors.

  • A tuple (x_val, y_val, val_sample_weights) of NumPy arrays.

  • A tf.data.Dataset.

  • A Python generator or keras.utils.Sequence returning

  (inputs, targets) or (inputs, targets, sample_weights).

  validation_data is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’). This argument is ignored when x is a generator or an object of tf.data.Dataset. ‘batch’ is a special option for dealing with the limitations of HDF5 data; it shuffles in batch-sized chunks. Has no effect when steps_per_epoch is not None.

• class_weight (Optional dictionary mapping class indices (integers)) – to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to “pay more attention” to samples from an under-represented class.

• sample_weight (Optional Numpy array of weights for) –

  the training samples, used for weighting the loss function

  (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. This argument is not supported when x is a dataset, generator, or

  keras.utils.Sequence instance, instead provide the sample_weights

  as the third element of x.

• initial_epoch (Integer.) – Epoch at which to start training (useful for resuming a previous training run).

• steps_per_epoch (Integer or None.) –

  Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. When training with input tensors such as TensorFlow data tensors, the default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. If x is a tf.data dataset, and ‘steps_per_epoch’ is None, the epoch will run until the input dataset is exhausted. When passing an infinitely repeating dataset, you must specify the steps_per_epoch argument. If steps_per_epoch=-1 the training will run indefinitely with an infinitely repeating dataset. This argument is not supported with array inputs. When using tf.distribute.experimental.ParameterServerStrategy:

  • steps_per_epoch=None is not supported.

• validation_steps (Only relevant if validation_data is provided and) – is a tf.data dataset. Total number of steps (batches of samples) to draw before stopping when performing validation at the end of every epoch. If ‘validation_steps’ is None, validation will run until the validation_data dataset is exhausted. In the case of an infinitely repeated dataset, it will run into an infinite loop. If ‘validation_steps’ is specified and only part of the dataset will be consumed, the evaluation will start from the beginning of the dataset at each epoch. This ensures that the same validation samples are used every time.

• validation_freq (Only relevant if validation data is provided. Integer) – or collections.abc.Container instance (e.g. list, tuple, etc.). If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. validation_freq=2 runs validation every 2 epochs. If a Container, specifies the epochs on which to run validation, e.g. validation_freq=[1, 2, 10] runs validation at the end of the 1st, 2nd, and 10th epochs.

• max_queue_size (Integer. Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

• workers (Integer. Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1.

• use_multiprocessing (Boolean. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes.

• details. (See tf.keras.model.Model.fit for) –

predict(X, batch_size=128)

Inference Function.

Task: time series classification or regression.

forecast(X)

Inference Function.

Task: time series forecast.

predict_proba(X, batch_size=128)

Inference Function.

Task: time series classification/regression.

proba2predict(proba, encode_to_label=True)

Transition Function.

Task: time series classification.

property monitor

Gets monitor for ReduceLROnPlateau and EarlyStopping.

property first_metric_name

Get first metric name.

plot_net(model_file)

Plot net model architecture.

save_model(model_file, name='dl_model', external=False)

Save the instance object.

static load_model(model_file, name='dl_model', external=False)

Load the instance object.

class hyperts.framework.dl._base.BaseDeepDetectionMixin(name=None, contamination=0.05)

Bases: object

Mixin class for anomaly detection in estimator wrapper.

Parameters:

• name (str, the name of detection algorithm.) –

• contamination (float, the range in (0., 0.5), optional (default=0.05).) – The amount of contamination of the data set, i.e. the proportion of outliers in the data set. Used when fitting to define the threshold on the decision function.

decision_scores_

The outlier scores of the training data.

Type:

numpy array of shape (n_samples,).

threshold_

It is the n_samples * contamination most abnormal samples in decision_scores_. The threshold is calculated for generating binary outlier labels.

Type:

float, the threshold is based on contamination.

labels_

The binary labels of the training data. 0 stands for inliers and 1 for outliers/anomalies. It is generated by applying threshold_ on decision_scores_.

Type:

int, either 0 or 1.

classes_

Default as binary classification.

Type:

int, default 2.

decision_function(X, y, batch_size=128, **kwargs)

Predict raw anomaly score of X using the fitted detector.

Parameters:

• X (numpy array of shape (n_samples, timestamp, continuous_dim) or) – [(n_samples, timestamp, continuous_dim), (n_samples, timestamp, categorical_dim)] The training input samples.

• y (numpy array of shape (n_samples, timestamp, reconstract_dim).) – The training input samples.

Returns:

anomaly_scores – The anomaly score of the input samples.

Return type:

numpy array of shape (n_samples,)

predict_outliers(X, y)

Predict labels for sequences in X.

Parameters:

• X (DataFrame of shape (n_samples, 1+(n_covariates)).) –

• y (DataFrame of shape (n_samples, reconstract_dim).) –

Returns:

outlier_labels – For each observation, tells whether or not it should be considered as an outlier according to the fitted model. 0 stands for inliers and 1 for outliers.

Return type:

numpy array of shape (n_samples,)

predict_outliers_prob(X, y, methed='erf')

Predict the probability for sequences in X.

Parameters:

• X (DataFrame of shape (n_samples, 1+(n_covariates)).) –

• y (DataFrame of shape (n_samples, reconstact_dim).) –

• methed (str, optional {'erf', 'linear'}. Probability conversion method.) –

Returns:

outlier_probability – For each observation, tells whether or not it should be considered as an outlier according to the fitted model. Return the outlier probability, ranging in [0,1]. Note it depends on the number of classes, which is by default 2 classes ([proba of normal, proba of outliers]).

Return type:

numpy array of shape (n_samples, n_classes)

predict_outliers_confidence(X, y)

Predict the confidence of model in making the same prediction

under slightly different training sets.

Parameters:

• X (DataFrame of shape (n_samples, 1+(n_covariates)).) –

• y (DataFrame of shape (n_samples, reconstact_dim).) –

Returns:

confidence – For each observation, tells how consistently the model would make the same prediction if the training set was perturbed. Return a probability, ranging in [0,1].

Return type:

numpy array of shape (n_samples,).

dl.models.deepar

hyperts.framework.dl.models.deepar.DeepARModel(task, window, rnn_type, continuous_columns, categorical_columns, rnn_units, rnn_layers, drop_rate=0.0, nb_outputs=1, nb_steps=1, out_activation='linear', **kwargs)

Deep AutoRegressive Model (DeepAR).

Parameters:

• task (Str - Only 'univariate-forecast' is supported.) –

• window (Positive Int - Length of the time series sequences for a sample,) –

• rnn_type (Str - Type of recurrent neural network, including) – optional {‘basic’, ‘gru’, ‘lstm}.

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

• rnn_units (Positive Int - The dimensionality of the output space for RNN.) –

• rnn_layers (Positive Int - The number of the layers for RNN.) –

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets.) –

• nb_outputs (Positive Int, Only and default 1.) –

• nb_steps (Positive Int, The step length of forecast, only and default 1.) –

• out_activation (Str - Forecast the task output activation function,) – optional {‘linear’, ‘sigmoid’, ‘tanh’}, default = ‘linear’.

class hyperts.framework.dl.models.deepar.DeepAR(task, timestamp, rnn_type='gru', rnn_units=16, rnn_layers=1, drop_rate=0.0, out_activation='linear', window=3, horizon=1, forecast_length=1, metrics='auto', monitor_metric='val_loss', optimizer='auto', learning_rate=0.001, loss='auto', reducelr_patience=5, earlystop_patience=10, summary=True, continuous_columns=None, categorical_columns=None, **kwargs)

Bases: BaseDeepEstimator

Deep AutoRegressive Estimator (DeepAR).

Parameters:

• task (Str - Only 'univariate-forecast' is supported,) – default = ‘univariate-forecast’.

• timestamp (Str - Timestamp name, not optional.) –

• rnn_type (Str - Type of recurrent neural network,) – {‘basic’, ‘gru’, ‘lstm}, default = ‘gru’.

• rnn_units (Positive Int - The dimensionality of the output space for recurrent neural network,) – default = 16.

• rnn_layers (Positive Int - The number of the layers for recurrent neural network,) – default = 1.

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets,) – default = 0.

• out_activation (Str - Forecast the task output activation function,) – optional {‘linear’, ‘sigmoid’, ‘tanh’}, default = ‘linear’.

• window (Positive Int - Length of the time series sequences for a sample,) – default = 3.

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor_metric (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Only 'log_gaussian_loss' is supported for DeepAR, which has been defined.) – default = ‘log_gaussian_loss’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

property first_metric_name

Get first metric name.

fit(X, y, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.2, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1, max_queue_size=10, workers=1, use_multiprocessing=False, **kwargs)

Trains the model for a fixed number of epochs (iterations on a dataset).

Parameters:

• X (2D DataFrame of shape: (series_length, 1+(n_covariates)) for forecast task,) – 2D nested DataFrame shape: (sample, nb_covariables(series_length)) for classification or regression task, 2D DataFrame of shape (n_samples, 1+(n_covariates)) for anomaly detection.

• y (2D DataFrame of shape: (series_length, n_target_variables) for forecast task,) – 2D DataFrame of shape: (nb_samples, 1) for classification or regression task, 2D DataFrame of shape (n_samples, reconstract_dim).

• batch_size (Integer or None.) – Number of samples per gradient update. If unspecified, batch_size will default to self-adaption based on number of samples. Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

• epochs (Integer. Number of epochs to train the model.) – An epoch is an iteration over the entire x and y data provided. Note that in conjunction with initial_epoch, epochs is to be understood as “final epoch”. The model is not trained for a number of iterations given by epochs, but merely until the epoch of index epochs is reached.

• verbose ('auto', 0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. ‘auto’ defaults to 1 for most cases, but 2 when used with ParameterServerStrategy. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment).

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. Callbacks with batch-level calls are currently unsupported with tf.distribute.experimental.ParameterServerStrategy, and users are advised to implement epoch-level calls instead with an appropriate steps_per_epoch value.

• validation_split (Float between 0 and 1.) –

  Fraction of the training data to be used as validation data.

  The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling. This argument is not supported when x is a dataset, generator or

  keras.utils.Sequence instance.

  validation_split is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• validation_data (Data on which to evaluate) –

  the loss and any model metrics at the end of each epoch. The model will not be trained on this data. Thus, note the fact that the validation loss of data provided using validation_split or validation_data is not affected by regularization layers like noise and dropout. validation_data will override validation_split. validation_data could be:

  • A tuple (x_val, y_val) of Numpy arrays or tensors.

  • A tuple (x_val, y_val, val_sample_weights) of NumPy arrays.

  • A tf.data.Dataset.

  • A Python generator or keras.utils.Sequence returning

  (inputs, targets) or (inputs, targets, sample_weights).

  validation_data is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’). This argument is ignored when x is a generator or an object of tf.data.Dataset. ‘batch’ is a special option for dealing with the limitations of HDF5 data; it shuffles in batch-sized chunks. Has no effect when steps_per_epoch is not None.

• class_weight (Optional dictionary mapping class indices (integers)) – to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to “pay more attention” to samples from an under-represented class.

• sample_weight (Optional Numpy array of weights for) –

  the training samples, used for weighting the loss function

  (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. This argument is not supported when x is a dataset, generator, or

  keras.utils.Sequence instance, instead provide the sample_weights

  as the third element of x.

• initial_epoch (Integer.) – Epoch at which to start training (useful for resuming a previous training run).

• steps_per_epoch (Integer or None.) –

  Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. When training with input tensors such as TensorFlow data tensors, the default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. If x is a tf.data dataset, and ‘steps_per_epoch’ is None, the epoch will run until the input dataset is exhausted. When passing an infinitely repeating dataset, you must specify the steps_per_epoch argument. If steps_per_epoch=-1 the training will run indefinitely with an infinitely repeating dataset. This argument is not supported with array inputs. When using tf.distribute.experimental.ParameterServerStrategy:

  • steps_per_epoch=None is not supported.

• validation_steps (Only relevant if validation_data is provided and) – is a tf.data dataset. Total number of steps (batches of samples) to draw before stopping when performing validation at the end of every epoch. If ‘validation_steps’ is None, validation will run until the validation_data dataset is exhausted. In the case of an infinitely repeated dataset, it will run into an infinite loop. If ‘validation_steps’ is specified and only part of the dataset will be consumed, the evaluation will start from the beginning of the dataset at each epoch. This ensures that the same validation samples are used every time.

• validation_freq (Only relevant if validation data is provided. Integer) – or collections.abc.Container instance (e.g. list, tuple, etc.). If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. validation_freq=2 runs validation every 2 epochs. If a Container, specifies the epochs on which to run validation, e.g. validation_freq=[1, 2, 10] runs validation at the end of the 1st, 2nd, and 10th epochs.

• max_queue_size (Integer. Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

• workers (Integer. Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1.

• use_multiprocessing (Boolean. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes.

• details. (See tf.keras.model.Model.fit for) –

forecast(X)

Inference Function.

Task: time series forecast.

static load_model(model_file, name='dl_model', external=False)

Load the instance object.

property monitor

Gets monitor for ReduceLROnPlateau and EarlyStopping.

plot_net(model_file)

Plot net model architecture.

predict(X, batch_size=128)

Inference Function.

Task: time series classification or regression.

predict_proba(X, batch_size=128)

Inference Function.

Task: time series classification/regression.

proba2predict(proba, encode_to_label=True)

Transition Function.

Task: time series classification.

save_model(model_file, name='dl_model', external=False)

Save the instance object.

dl.models.hybird_rnn

dl.models.inceptiontime

hyperts.framework.dl.models.inceptiontime.InceptionTimeModel(task, window, continuous_columns, categorical_columns, blocks=3, cnn_filters=32, bottleneck_size=32, kernel_size_list=(1, 3, 5, 8, 12), shortcut=True, short_filters=64, nb_outputs=1, **kwargs)

Inception Time Model (InceptionTime).

Parameters:

• task (Str - Only 'classification' is supported.) –

• window (Positive Int - Length of the time series sequences for a sample.) –

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

• blocks (Int - The depth of the net architecture.) –

• cnn_filters (Int - The number of cnn filters.) –

• bottleneck_size (Int - The number of bottleneck (a cnn layer).) –

• kernel_size_list (Tuple - The kernel size of cnn for a inceptionblock.) –

• shortcut (Bool - Whether to use shortcut opration.) –

• short_filters (Int - The number of filters of shortcut conv1d layer.) –

• nb_outputs (Int - The number of classes, default 1.) –

class hyperts.framework.dl.models.inceptiontime.InceptionTime(task, blocks=3, cnn_filters=32, bottleneck_size=32, kernel_size_list=(1, 3, 5, 8, 12), shortcut=True, short_filters=64, timestamp=None, window=3, horizon=1, forecast_length=1, metrics='auto', monitor_metric='val_loss', optimizer='auto', learning_rate=0.001, loss='auto', reducelr_patience=5, earlystop_patience=10, summary=True, continuous_columns=None, categorical_columns=None, **kwargs)

Bases: BaseDeepEstimator

InceptionTime Estimator.

Parameters:

• task (Str - Support forecast, classification, and regression.) – See hyperts.utils.consts for details.

• blocks (Int - The depth of the net architecture.) – default = 3.

• cnn_filters (Int - The number of cnn filters.) – default = 32.

• bottleneck_size (Int - The number of bottleneck (a cnn layer).) – default = 32.

• kernel_size_list (Tuple - The kernel size of cnn for a inceptionblock.) – default = (1, 3, 5, 8, 12).

• shortcut (Bool - Whether to use shortcut opration.) – default = True.

• short_filters (Int - The number of filters of shortcut conv1d layer.) – default = 64.

• timestamp (Str or None - Timestamp name, the forecast task must be given,) – default None.

• window (Positive Int - Length of the time series sequences for a sample,) – default = 3.

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor_metric (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Loss function, for forecsting or regression, optional {'auto', 'mae', 'mse', 'huber_loss',) – ‘mape’}, for classification, optional {‘auto’, ‘categorical_crossentropy’, ‘binary_crossentropy}, default = ‘auto’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

property first_metric_name

Get first metric name.

fit(X, y, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.2, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1, max_queue_size=10, workers=1, use_multiprocessing=False, **kwargs)

Trains the model for a fixed number of epochs (iterations on a dataset).

Parameters:

• X (2D DataFrame of shape: (series_length, 1+(n_covariates)) for forecast task,) – 2D nested DataFrame shape: (sample, nb_covariables(series_length)) for classification or regression task, 2D DataFrame of shape (n_samples, 1+(n_covariates)) for anomaly detection.

• y (2D DataFrame of shape: (series_length, n_target_variables) for forecast task,) – 2D DataFrame of shape: (nb_samples, 1) for classification or regression task, 2D DataFrame of shape (n_samples, reconstract_dim).

• batch_size (Integer or None.) – Number of samples per gradient update. If unspecified, batch_size will default to self-adaption based on number of samples. Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

• epochs (Integer. Number of epochs to train the model.) – An epoch is an iteration over the entire x and y data provided. Note that in conjunction with initial_epoch, epochs is to be understood as “final epoch”. The model is not trained for a number of iterations given by epochs, but merely until the epoch of index epochs is reached.

• verbose ('auto', 0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. ‘auto’ defaults to 1 for most cases, but 2 when used with ParameterServerStrategy. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment).

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. Callbacks with batch-level calls are currently unsupported with tf.distribute.experimental.ParameterServerStrategy, and users are advised to implement epoch-level calls instead with an appropriate steps_per_epoch value.

• validation_split (Float between 0 and 1.) –

  Fraction of the training data to be used as validation data.

  The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling. This argument is not supported when x is a dataset, generator or

  keras.utils.Sequence instance.

  validation_split is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• validation_data (Data on which to evaluate) –

  the loss and any model metrics at the end of each epoch. The model will not be trained on this data. Thus, note the fact that the validation loss of data provided using validation_split or validation_data is not affected by regularization layers like noise and dropout. validation_data will override validation_split. validation_data could be:

  • A tuple (x_val, y_val) of Numpy arrays or tensors.

  • A tuple (x_val, y_val, val_sample_weights) of NumPy arrays.

  • A tf.data.Dataset.

  • A Python generator or keras.utils.Sequence returning

  (inputs, targets) or (inputs, targets, sample_weights).

  validation_data is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’). This argument is ignored when x is a generator or an object of tf.data.Dataset. ‘batch’ is a special option for dealing with the limitations of HDF5 data; it shuffles in batch-sized chunks. Has no effect when steps_per_epoch is not None.

• class_weight (Optional dictionary mapping class indices (integers)) – to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to “pay more attention” to samples from an under-represented class.

• sample_weight (Optional Numpy array of weights for) –

  the training samples, used for weighting the loss function

  (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. This argument is not supported when x is a dataset, generator, or

  keras.utils.Sequence instance, instead provide the sample_weights

  as the third element of x.

• initial_epoch (Integer.) – Epoch at which to start training (useful for resuming a previous training run).

• steps_per_epoch (Integer or None.) –

  Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. When training with input tensors such as TensorFlow data tensors, the default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. If x is a tf.data dataset, and ‘steps_per_epoch’ is None, the epoch will run until the input dataset is exhausted. When passing an infinitely repeating dataset, you must specify the steps_per_epoch argument. If steps_per_epoch=-1 the training will run indefinitely with an infinitely repeating dataset. This argument is not supported with array inputs. When using tf.distribute.experimental.ParameterServerStrategy:

  • steps_per_epoch=None is not supported.

• validation_steps (Only relevant if validation_data is provided and) – is a tf.data dataset. Total number of steps (batches of samples) to draw before stopping when performing validation at the end of every epoch. If ‘validation_steps’ is None, validation will run until the validation_data dataset is exhausted. In the case of an infinitely repeated dataset, it will run into an infinite loop. If ‘validation_steps’ is specified and only part of the dataset will be consumed, the evaluation will start from the beginning of the dataset at each epoch. This ensures that the same validation samples are used every time.

• validation_freq (Only relevant if validation data is provided. Integer) – or collections.abc.Container instance (e.g. list, tuple, etc.). If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. validation_freq=2 runs validation every 2 epochs. If a Container, specifies the epochs on which to run validation, e.g. validation_freq=[1, 2, 10] runs validation at the end of the 1st, 2nd, and 10th epochs.

• max_queue_size (Integer. Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

• workers (Integer. Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1.

• use_multiprocessing (Boolean. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes.

• details. (See tf.keras.model.Model.fit for) –

forecast(X)

Inference Function.

Task: time series forecast.

static load_model(model_file, name='dl_model', external=False)

Load the instance object.

property monitor

Gets monitor for ReduceLROnPlateau and EarlyStopping.

plot_net(model_file)

Plot net model architecture.

predict(X, batch_size=128)

Inference Function.

Task: time series classification or regression.

predict_proba(X, batch_size=128)

Inference Function.

Task: time series classification/regression.

proba2predict(proba, encode_to_label=True)

Transition Function.

Task: time series classification.

save_model(model_file, name='dl_model', external=False)

Save the instance object.

dl.models.lstnet

hyperts.framework.dl.models.lstnet.LSTNetModel(task, window, rnn_type, skip_rnn_type, continuous_columns, categorical_columns, cnn_filters, kernel_size, rnn_units, rnn_layers, skip_rnn_units, skip_rnn_layers, skip_period, ar_order, drop_rate=0.0, nb_outputs=1, nb_steps=1, out_activation='linear', **kwargs)

Long-and Short-term Time-series Network Model (LSTNet).

Parameters:

• task (Str - Support forecast, classification, and regression.) – See hyperts.utils.consts for details.

• window (Positive Int - Length of the time series sequences for a sample.) –

• rnn_type (Str - Type of recurrent neural network,) – optional {‘basic’, ‘gru’, ‘lstm}.

• skip_rnn_type (Str - Type of skip recurrent neural network,) – optional {‘simple_rnn’, ‘gru’, ‘lstm}, default = ‘gru’.

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

• cnn_filters (Positive Int - The dimensionality of the output space (i.e. the number of filters) – in the convolution).

• kernel_size (Positive Int - A single integer specifying the spatial dimensions of the filters,) –

• rnn_units (Positive Int - The dimensionality of the output space for RNN.) –

• rnn_layers (Positive Int - The number of the layers for RNN.) –

• skip_rnn_units (Positive Int - The dimensionality of the output space for skip RNN.) –

• skip_rnn_layers (Positive Int - The number of the layers for skip RNN.) –

• skip_period (Positive Int or 0 - The length of skip for recurrent neural network.) –

• ar_order (Positive Int or None - The window size of the autoregressive component.) –

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets.) –

• nb_outputs (Int, default 1.) –

• nb_steps (Int, The step length of forecast, default 1.) –

• out_activation (Str - Forecast the task output activation function,) – optional {‘linear’, ‘sigmoid’, ‘tanh’}, default = ‘linear’.

class hyperts.framework.dl.models.lstnet.LSTNet(task, rnn_type='gru', skip_rnn_type='gru', cnn_filters=16, kernel_size=1, rnn_units=16, rnn_layers=1, skip_rnn_units=16, skip_rnn_layers=1, skip_period=0, ar_order=0, drop_rate=0.0, out_activation='linear', timestamp=None, window=7, horizon=1, forecast_length=1, metrics='auto', monitor_metric='val_loss', optimizer='auto', learning_rate=0.001, loss='auto', reducelr_patience=5, earlystop_patience=10, summary=True, continuous_columns=None, categorical_columns=None, **kwargs)

Bases: BaseDeepEstimator

Long-and Short-term Time-series Network Estimator (LSTNet).

Parameters:

• task (Str - Support forecast, classification, and regression.) – See hyperts.utils.consts for details.

• rnn_type (Str - Type of recurrent neural network,) – optional {‘simple_rnn’, ‘gru’, ‘lstm}, default = ‘gru’.

• skip_rnn_type (Str - Type of skip recurrent neural network,) – optional {‘basic’, ‘gru’, ‘lstm}, default = ‘gru’.

• cnn_filters (Positive Int - The dimensionality of the output space (i.e. the number of filters) – in the convolution), default = 16.

• kernel_size (Positive Int - A single integer specifying the spatial dimensions of the filters,) – default = 1.

• rnn_units (Positive Int - The dimensionality of the output space for recurrent neural network,) – default = 16.

• rnn_layers (Positive Int - The number of the layers for recurrent neural network,) – default = 1.

• skip_rnn_units (Positive Int - The dimensionality of the output space for skip recurrent neural network,) – default = 16.

• skip_rnn_layers (Positive Int - The number of the layers for skip recurrent neural network,) – default = 1.

• skip_period (Positive Int or 0 - The length of skip for recurrent neural network,) – default = 0.

• ar_order (Positive Int or None - The window size of the autoregressive component,) – default = None.

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets,) – default = 0.

• out_activation (Str - Forecast the task output activation function, optional {'linear', 'sigmoid', 'tanh'},) – default = ‘linear’.

• timestamp (Str or None - Timestamp name, the forecast task must be given,) – default None.

• window (Positive Int - Length of the time series sequences for a sample,) – default = 7.

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor_metric (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Loss function, for forecsting or regression, optional {'auto', 'mae', 'mse', 'huber_loss',) – ‘mape’}, for classification, optional {‘auto’, ‘categorical_crossentropy’, ‘binary_crossentropy}, default = ‘auto’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

property first_metric_name

Get first metric name.

fit(X, y, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.2, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1, max_queue_size=10, workers=1, use_multiprocessing=False, **kwargs)

Trains the model for a fixed number of epochs (iterations on a dataset).

Parameters:

• X (2D DataFrame of shape: (series_length, 1+(n_covariates)) for forecast task,) – 2D nested DataFrame shape: (sample, nb_covariables(series_length)) for classification or regression task, 2D DataFrame of shape (n_samples, 1+(n_covariates)) for anomaly detection.

• y (2D DataFrame of shape: (series_length, n_target_variables) for forecast task,) – 2D DataFrame of shape: (nb_samples, 1) for classification or regression task, 2D DataFrame of shape (n_samples, reconstract_dim).

• batch_size (Integer or None.) – Number of samples per gradient update. If unspecified, batch_size will default to self-adaption based on number of samples. Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

• epochs (Integer. Number of epochs to train the model.) – An epoch is an iteration over the entire x and y data provided. Note that in conjunction with initial_epoch, epochs is to be understood as “final epoch”. The model is not trained for a number of iterations given by epochs, but merely until the epoch of index epochs is reached.

• verbose ('auto', 0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. ‘auto’ defaults to 1 for most cases, but 2 when used with ParameterServerStrategy. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment).

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. Callbacks with batch-level calls are currently unsupported with tf.distribute.experimental.ParameterServerStrategy, and users are advised to implement epoch-level calls instead with an appropriate steps_per_epoch value.

• validation_split (Float between 0 and 1.) –

  Fraction of the training data to be used as validation data.

  The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling. This argument is not supported when x is a dataset, generator or

  keras.utils.Sequence instance.

  validation_split is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• validation_data (Data on which to evaluate) –

  the loss and any model metrics at the end of each epoch. The model will not be trained on this data. Thus, note the fact that the validation loss of data provided using validation_split or validation_data is not affected by regularization layers like noise and dropout. validation_data will override validation_split. validation_data could be:

  • A tuple (x_val, y_val) of Numpy arrays or tensors.

  • A tuple (x_val, y_val, val_sample_weights) of NumPy arrays.

  • A tf.data.Dataset.

  • A Python generator or keras.utils.Sequence returning

  (inputs, targets) or (inputs, targets, sample_weights).

  validation_data is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’). This argument is ignored when x is a generator or an object of tf.data.Dataset. ‘batch’ is a special option for dealing with the limitations of HDF5 data; it shuffles in batch-sized chunks. Has no effect when steps_per_epoch is not None.

• class_weight (Optional dictionary mapping class indices (integers)) – to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to “pay more attention” to samples from an under-represented class.

• sample_weight (Optional Numpy array of weights for) –

  the training samples, used for weighting the loss function

  (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. This argument is not supported when x is a dataset, generator, or

  keras.utils.Sequence instance, instead provide the sample_weights

  as the third element of x.

• initial_epoch (Integer.) – Epoch at which to start training (useful for resuming a previous training run).

• steps_per_epoch (Integer or None.) –

  Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. When training with input tensors such as TensorFlow data tensors, the default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. If x is a tf.data dataset, and ‘steps_per_epoch’ is None, the epoch will run until the input dataset is exhausted. When passing an infinitely repeating dataset, you must specify the steps_per_epoch argument. If steps_per_epoch=-1 the training will run indefinitely with an infinitely repeating dataset. This argument is not supported with array inputs. When using tf.distribute.experimental.ParameterServerStrategy:

  • steps_per_epoch=None is not supported.

• validation_steps (Only relevant if validation_data is provided and) – is a tf.data dataset. Total number of steps (batches of samples) to draw before stopping when performing validation at the end of every epoch. If ‘validation_steps’ is None, validation will run until the validation_data dataset is exhausted. In the case of an infinitely repeated dataset, it will run into an infinite loop. If ‘validation_steps’ is specified and only part of the dataset will be consumed, the evaluation will start from the beginning of the dataset at each epoch. This ensures that the same validation samples are used every time.

• validation_freq (Only relevant if validation data is provided. Integer) – or collections.abc.Container instance (e.g. list, tuple, etc.). If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. validation_freq=2 runs validation every 2 epochs. If a Container, specifies the epochs on which to run validation, e.g. validation_freq=[1, 2, 10] runs validation at the end of the 1st, 2nd, and 10th epochs.

• max_queue_size (Integer. Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

• workers (Integer. Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1.

• use_multiprocessing (Boolean. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes.

• details. (See tf.keras.model.Model.fit for) –

forecast(X)

Inference Function.

Task: time series forecast.

static load_model(model_file, name='dl_model', external=False)

Load the instance object.

property monitor

Gets monitor for ReduceLROnPlateau and EarlyStopping.

plot_net(model_file)

Plot net model architecture.

predict(X, batch_size=128)

Inference Function.

Task: time series classification or regression.

predict_proba(X, batch_size=128)

Inference Function.

Task: time series classification/regression.

proba2predict(proba, encode_to_label=True)

Transition Function.

Task: time series classification.

save_model(model_file, name='dl_model', external=False)

Save the instance object.

dl.models.nbeats

hyperts.framework.dl.models.nbeats.linear_space(backcast_length, forecast_length, is_forecast=True)

hyperts.framework.dl.models.nbeats.NBeatsModel(task, continuous_columns, categorical_columns, window=10, nb_steps=1, stack_types=('trend', 'seasonality'), thetas_dim=(4, 8), nb_blocks_per_stack=3, share_weights_in_stack=False, hidden_layer_units=256, nb_outputs=1, nb_harmonics=None, out_activation='linear', **kwargs)

N-BEATS: Neural basis expansion analysis for interpretable time series forecasting.

Parameters:

• task (Str - Support forecast, classification, and regression.) – See hyperts.utils.consts for details.

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

• window (Positive Int - Length of the time series sequences for a sample,) – i.e., backcast_length, default 10.

• nb_steps (Positive Int - The step length of forecast, i.e., forecast_length, default 1.) –

• stack_types (Tuple(Str) - Stack types, optional {'trend', 'seasonality', 'generic'}.) –

• thetas_dim (Tuple(Int) - The number of units that make up each dense layer in each) – block of every stack.

• nb_blocks_per_stack (Int - The number of block per stack.) –

• share_weights_in_stack (Bool - Whether to share weights in stack.) –

• hidden_layer_units (Int - The units of hidden layer.) –

• nb_outputs (Int, default 1.) –

• nb_harmonics (Int or None, -The number of harmonic terms for each stack type, default None.) –

• out_activation (Str - Forecast the task output activation function,) – optional {‘linear’, ‘sigmoid’}, default = ‘linear’.

References

[1] Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, Yoshua Bengio, N-BEATS: Neural basis

expansion analysis for interpretable time series forecasting, ICLR 2020.

[2] https://github.com/philipperemy/n-beats.

class hyperts.framework.dl.models.nbeats.NBeats(task, stack_types=('trend', 'seasonality'), thetas_dim=(4, 8), nb_blocks_per_stack=3, share_weights_in_stack=False, hidden_layer_units=256, out_activation='linear', timestamp=None, window=3, horizon=1, forecast_length=1, metrics='auto', monitor_metric='val_loss', optimizer='auto', learning_rate=0.001, loss='auto', reducelr_patience=5, earlystop_patience=10, summary=True, continuous_columns=None, categorical_columns=None, **kwargs)

Bases: BaseDeepEstimator

NBeats Estimator .

Parameters:

• task (Str - Support forecast, classification, and regression.) – See hyperts.utils.consts for details.

• stack_types (Tuple(Str) - Stack types, optional {'trend', 'seasonality', 'generic'}.) – default = (‘trend’, ‘seasonality’).

• thetas_dim (Tuple(Int) - The number of units that make up each dense layer in each block of every stack.) – default = (4, 8).

• nb_blocks_per_stack (Int - The number of block per stack.) – default = 3.

• share_weights_in_stack (Bool - Whether to share weights in stack.) – default = False.

• hidden_layer_units (Int - The units of hidden layer.) – default = 256.

• out_activation (Str - Forecast the task output activation function, optional {'linear', 'sigmoid', 'tanh'},) – default = ‘linear’.

• timestamp (Str or None - Timestamp name, the forecast task must be given,) – default None.

• window (Positive Int - Length of the time series sequences for a sample,) – default = 3.

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor_metric (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Loss function, for forecsting or regression, optional {'auto', 'mae', 'mse', 'huber_loss',) – ‘mape’}, for classification, optional {‘auto’, ‘categorical_crossentropy’, ‘binary_crossentropy}, default = ‘auto’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

property first_metric_name

Get first metric name.

fit(X, y, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.2, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1, max_queue_size=10, workers=1, use_multiprocessing=False, **kwargs)

Trains the model for a fixed number of epochs (iterations on a dataset).

Parameters:

• X (2D DataFrame of shape: (series_length, 1+(n_covariates)) for forecast task,) – 2D nested DataFrame shape: (sample, nb_covariables(series_length)) for classification or regression task, 2D DataFrame of shape (n_samples, 1+(n_covariates)) for anomaly detection.

• y (2D DataFrame of shape: (series_length, n_target_variables) for forecast task,) – 2D DataFrame of shape: (nb_samples, 1) for classification or regression task, 2D DataFrame of shape (n_samples, reconstract_dim).

• batch_size (Integer or None.) – Number of samples per gradient update. If unspecified, batch_size will default to self-adaption based on number of samples. Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

• epochs (Integer. Number of epochs to train the model.) – An epoch is an iteration over the entire x and y data provided. Note that in conjunction with initial_epoch, epochs is to be understood as “final epoch”. The model is not trained for a number of iterations given by epochs, but merely until the epoch of index epochs is reached.

• verbose ('auto', 0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. ‘auto’ defaults to 1 for most cases, but 2 when used with ParameterServerStrategy. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment).

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. Callbacks with batch-level calls are currently unsupported with tf.distribute.experimental.ParameterServerStrategy, and users are advised to implement epoch-level calls instead with an appropriate steps_per_epoch value.

• validation_split (Float between 0 and 1.) –

  Fraction of the training data to be used as validation data.

  The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling. This argument is not supported when x is a dataset, generator or

  keras.utils.Sequence instance.

  validation_split is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• validation_data (Data on which to evaluate) –

  the loss and any model metrics at the end of each epoch. The model will not be trained on this data. Thus, note the fact that the validation loss of data provided using validation_split or validation_data is not affected by regularization layers like noise and dropout. validation_data will override validation_split. validation_data could be:

  • A tuple (x_val, y_val) of Numpy arrays or tensors.

  • A tuple (x_val, y_val, val_sample_weights) of NumPy arrays.

  • A tf.data.Dataset.

  • A Python generator or keras.utils.Sequence returning

  (inputs, targets) or (inputs, targets, sample_weights).

  validation_data is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’). This argument is ignored when x is a generator or an object of tf.data.Dataset. ‘batch’ is a special option for dealing with the limitations of HDF5 data; it shuffles in batch-sized chunks. Has no effect when steps_per_epoch is not None.

• class_weight (Optional dictionary mapping class indices (integers)) – to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to “pay more attention” to samples from an under-represented class.

• sample_weight (Optional Numpy array of weights for) –

  the training samples, used for weighting the loss function

  (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. This argument is not supported when x is a dataset, generator, or

  keras.utils.Sequence instance, instead provide the sample_weights

  as the third element of x.

• initial_epoch (Integer.) – Epoch at which to start training (useful for resuming a previous training run).

• steps_per_epoch (Integer or None.) –

  Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. When training with input tensors such as TensorFlow data tensors, the default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. If x is a tf.data dataset, and ‘steps_per_epoch’ is None, the epoch will run until the input dataset is exhausted. When passing an infinitely repeating dataset, you must specify the steps_per_epoch argument. If steps_per_epoch=-1 the training will run indefinitely with an infinitely repeating dataset. This argument is not supported with array inputs. When using tf.distribute.experimental.ParameterServerStrategy:

  • steps_per_epoch=None is not supported.

• validation_steps (Only relevant if validation_data is provided and) – is a tf.data dataset. Total number of steps (batches of samples) to draw before stopping when performing validation at the end of every epoch. If ‘validation_steps’ is None, validation will run until the validation_data dataset is exhausted. In the case of an infinitely repeated dataset, it will run into an infinite loop. If ‘validation_steps’ is specified and only part of the dataset will be consumed, the evaluation will start from the beginning of the dataset at each epoch. This ensures that the same validation samples are used every time.

• validation_freq (Only relevant if validation data is provided. Integer) – or collections.abc.Container instance (e.g. list, tuple, etc.). If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. validation_freq=2 runs validation every 2 epochs. If a Container, specifies the epochs on which to run validation, e.g. validation_freq=[1, 2, 10] runs validation at the end of the 1st, 2nd, and 10th epochs.

• max_queue_size (Integer. Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

• workers (Integer. Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1.

• use_multiprocessing (Boolean. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes.

• details. (See tf.keras.model.Model.fit for) –

forecast(X)

Inference Function.

Task: time series forecast.

static load_model(model_file, name='dl_model', external=False)

Load the instance object.

property monitor

Gets monitor for ReduceLROnPlateau and EarlyStopping.

plot_net(model_file)

Plot net model architecture.

predict(X, batch_size=128)

Inference Function.

Task: time series classification or regression.

predict_proba(X, batch_size=128)

Inference Function.

Task: time series classification/regression.

proba2predict(proba, encode_to_label=True)

Transition Function.

Task: time series classification.

save_model(model_file, name='dl_model', external=False)

Save the instance object.

dl.models.vae

hyperts.framework.dl.models.vae.vae_loss_funcion(input_x, decoder, z_mean, z_log_var, reconstruct_loss='mse', beta=0.1)

Loss = Reconstruction loss + Kullback-Leibler loss

Parameters:

• input_x (Tensor, input time series.) – shape (batch_size, timestamp, features).

• decoder (Tensor, decoder reconstruct output.) – shape (batch_size, timestamp, features).

• z_mean (Tensor, latent representation mean of encoder.) – shape (batch_size, latent_dim).

• z_log_var (Tensor, latent representation log_var of encoder.) – shape (batch_size, latent_dim).

• reconstruct_loss (str, optional {'binary_crossentropy', ''mse'}.) –

• beta (positive float, a coefficient.) –

hyperts.framework.dl.models.vae.ConvVAEModel(task, window, nb_outputs, continuous_columns, categorical_columns, latent_dim=2, conv_type='general', cnn_filters=16, kernel_size=1, strides=1, nb_layers=2, activation='relu', drop_rate=0.0, out_activation='linear', **kwargs)

Variational AutoEncoder (VAE).

Parameters:

• task (Str - Only support anomaly detection.) – See hyperts.utils.consts for details.

• window (Positive Int - Length of the time series sequences for a sample.) –

• nb_outputs (Int, default 1.) –

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

• latent_dim (Int - Latent representation of encoder, default 2.) –

• conv_type (Str - Type of 1D convolution, optional {'general', 'separable'},) – default ‘general’.

• cnn_filters (Positive Int - The dimensionality of the output space (i.e. the number) – of filters in the convolution).

• kernel_size (Positive Int - A single integer specifying the spatial dimensions) – of the filters,

• strides (Int or tuple/list of a single integer - Specifying the stride length) – of the convolution.

• nb_layers (Int - The layers of encoder and decoder, default 2.) –

• activation (Str - The activation of hidden layers, default 'relu'.) –

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets.) –

• out_activation (Str - Forecast the task output activation function,) – optional {‘linear’, ‘sigmoid’, ‘tanh’}, default ‘linear’.

class hyperts.framework.dl.models.vae.ConvVAE(task, timestamp, contamination=0.05, window=3, horizon=1, forecast_length=1, latent_dim=2, conv_type='separable', cnn_filters=16, kernel_size=1, strides=1, nb_layers=2, activation='relu', drop_rate=0.2, out_activation='linear', reconstract_dim=None, metrics='auto', monitor_metric='val_loss', optimizer='auto', learning_rate=0.001, reducelr_patience=5, earlystop_patience=10, summary=True, continuous_columns=None, categorical_columns=None, name='conv_vae', **kwargs)

Bases: BaseDeepEstimator, BaseDeepDetectionMixin

Convolution Variational AutoEncoder Estimator (VAE).

Parameters:

• task (Str - Only support anomaly detection.) – See hyperts.utils.consts for details.

• timestamp (Str or None - Timestamp name, the forecast task must be given,) – default None.

• window (Positive Int - Length of the time series sequences for a sample.) –

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• latent_dim (Int - Latent representation of encoder, default 2.) –

• conv_type (Str - Type of 1D convolution, optional {'general', 'separable'},) – default ‘general’.

• cnn_filters (Positive Int - The dimensionality of the output space (i.e. the number) – of filters in the convolution).

• kernel_size (Positive Int - A single integer specifying the spatial dimensions) – of the filters,

• strides (Int or tuple/list of a single integer - Specifying the stride length) – of the convolution.

• nb_layers (Int - The layers of encoder and decoder, default 2.) –

• activation (Str - The activation of hidden layers, default 'relu'.) –

• drop_rate (Float between 0 and 1 - The rate of Dropout for neural nets.) –

• out_activation (Str - Forecast the task output activation function,) – optional {‘linear’, ‘sigmoid’, ‘tanh’}, default ‘linear’.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor_metric (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

decision_function(X, y, batch_size=128, **kwargs)

Predict raw anomaly score of X using the fitted detector.

Parameters:

• X (numpy array of shape (n_samples, timestamp, continuous_dim) or) – [(n_samples, timestamp, continuous_dim), (n_samples, timestamp, categorical_dim)] The training input samples.

• y (numpy array of shape (n_samples, timestamp, reconstract_dim).) – The training input samples.

Returns:

anomaly_scores – The anomaly score of the input samples.

Return type:

numpy array of shape (n_samples,)

property first_metric_name

Get first metric name.

fit(X, y, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.2, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1, max_queue_size=10, workers=1, use_multiprocessing=False, **kwargs)

Trains the model for a fixed number of epochs (iterations on a dataset).

Parameters:

• X (2D DataFrame of shape: (series_length, 1+(n_covariates)) for forecast task,) – 2D nested DataFrame shape: (sample, nb_covariables(series_length)) for classification or regression task, 2D DataFrame of shape (n_samples, 1+(n_covariates)) for anomaly detection.

• y (2D DataFrame of shape: (series_length, n_target_variables) for forecast task,) – 2D DataFrame of shape: (nb_samples, 1) for classification or regression task, 2D DataFrame of shape (n_samples, reconstract_dim).

• batch_size (Integer or None.) – Number of samples per gradient update. If unspecified, batch_size will default to self-adaption based on number of samples. Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

• epochs (Integer. Number of epochs to train the model.) – An epoch is an iteration over the entire x and y data provided. Note that in conjunction with initial_epoch, epochs is to be understood as “final epoch”. The model is not trained for a number of iterations given by epochs, but merely until the epoch of index epochs is reached.

• verbose ('auto', 0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. ‘auto’ defaults to 1 for most cases, but 2 when used with ParameterServerStrategy. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment).

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. Callbacks with batch-level calls are currently unsupported with tf.distribute.experimental.ParameterServerStrategy, and users are advised to implement epoch-level calls instead with an appropriate steps_per_epoch value.

• validation_split (Float between 0 and 1.) –

  Fraction of the training data to be used as validation data.

  The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling. This argument is not supported when x is a dataset, generator or

  keras.utils.Sequence instance.

  validation_split is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• validation_data (Data on which to evaluate) –

  the loss and any model metrics at the end of each epoch. The model will not be trained on this data. Thus, note the fact that the validation loss of data provided using validation_split or validation_data is not affected by regularization layers like noise and dropout. validation_data will override validation_split. validation_data could be:

  • A tuple (x_val, y_val) of Numpy arrays or tensors.

  • A tuple (x_val, y_val, val_sample_weights) of NumPy arrays.

  • A tf.data.Dataset.

  • A Python generator or keras.utils.Sequence returning

  (inputs, targets) or (inputs, targets, sample_weights).

  validation_data is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’). This argument is ignored when x is a generator or an object of tf.data.Dataset. ‘batch’ is a special option for dealing with the limitations of HDF5 data; it shuffles in batch-sized chunks. Has no effect when steps_per_epoch is not None.

• class_weight (Optional dictionary mapping class indices (integers)) – to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to “pay more attention” to samples from an under-represented class.

• sample_weight (Optional Numpy array of weights for) –

  the training samples, used for weighting the loss function

  (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. This argument is not supported when x is a dataset, generator, or

  keras.utils.Sequence instance, instead provide the sample_weights

  as the third element of x.

• initial_epoch (Integer.) – Epoch at which to start training (useful for resuming a previous training run).

• steps_per_epoch (Integer or None.) –

  Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. When training with input tensors such as TensorFlow data tensors, the default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. If x is a tf.data dataset, and ‘steps_per_epoch’ is None, the epoch will run until the input dataset is exhausted. When passing an infinitely repeating dataset, you must specify the steps_per_epoch argument. If steps_per_epoch=-1 the training will run indefinitely with an infinitely repeating dataset. This argument is not supported with array inputs. When using tf.distribute.experimental.ParameterServerStrategy:

  • steps_per_epoch=None is not supported.

• validation_steps (Only relevant if validation_data is provided and) – is a tf.data dataset. Total number of steps (batches of samples) to draw before stopping when performing validation at the end of every epoch. If ‘validation_steps’ is None, validation will run until the validation_data dataset is exhausted. In the case of an infinitely repeated dataset, it will run into an infinite loop. If ‘validation_steps’ is specified and only part of the dataset will be consumed, the evaluation will start from the beginning of the dataset at each epoch. This ensures that the same validation samples are used every time.

• validation_freq (Only relevant if validation data is provided. Integer) – or collections.abc.Container instance (e.g. list, tuple, etc.). If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. validation_freq=2 runs validation every 2 epochs. If a Container, specifies the epochs on which to run validation, e.g. validation_freq=[1, 2, 10] runs validation at the end of the 1st, 2nd, and 10th epochs.

• max_queue_size (Integer. Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

• workers (Integer. Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1.

• use_multiprocessing (Boolean. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes.

• details. (See tf.keras.model.Model.fit for) –

forecast(X)

Inference Function.

Task: time series forecast.

static load_model(model_file, name='dl_model', external=False)

Load the instance object.

property monitor

Gets monitor for ReduceLROnPlateau and EarlyStopping.

plot_net(model_file)

Plot net model architecture.

predict(X, batch_size=128)

Inference Function.

Task: time series classification or regression.

predict_outliers(X, y)

Predict labels for sequences in X.

Parameters:

• X (DataFrame of shape (n_samples, 1+(n_covariates)).) –

• y (DataFrame of shape (n_samples, reconstract_dim).) –

Returns:

outlier_labels – For each observation, tells whether or not it should be considered as an outlier according to the fitted model. 0 stands for inliers and 1 for outliers.

Return type:

numpy array of shape (n_samples,)

predict_outliers_confidence(X, y)

Predict the confidence of model in making the same prediction

under slightly different training sets.

Parameters:

• X (DataFrame of shape (n_samples, 1+(n_covariates)).) –

• y (DataFrame of shape (n_samples, reconstact_dim).) –

Returns:

confidence – For each observation, tells how consistently the model would make the same prediction if the training set was perturbed. Return a probability, ranging in [0,1].

Return type:

numpy array of shape (n_samples,).

predict_outliers_prob(X, y, methed='erf')

Predict the probability for sequences in X.

Parameters:

• X (DataFrame of shape (n_samples, 1+(n_covariates)).) –

• y (DataFrame of shape (n_samples, reconstact_dim).) –

• methed (str, optional {'erf', 'linear'}. Probability conversion method.) –

Returns:

outlier_probability – For each observation, tells whether or not it should be considered as an outlier according to the fitted model. Return the outlier probability, ranging in [0,1]. Note it depends on the number of classes, which is by default 2 classes ([proba of normal, proba of outliers]).

Return type:

numpy array of shape (n_samples, n_classes)

predict_proba(X, batch_size=128)

Inference Function.

Task: time series classification/regression.

proba2predict(proba, encode_to_label=True)

Transition Function.

Task: time series classification.

save_model(model_file, name='dl_model', external=False)

Save the instance object.

dl.layers

class hyperts.framework.dl.layers._layers.MultiColEmbedding(*args, **kwargs)

Bases: Layer

Multi Columns Embedding base on Embedding.

Parameters:

• input_dims (Integer. A list or tuple. Sizes of the vocabulary for per columns.) –

• output_dims (A list or tuple. Dimension of the dense embedding.) –

• input_length (Integer. Length of input sequences.) –

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.WeightedAttention(*args, **kwargs)

Bases: Layer

Weighted Attention based on softmax-Dense.

Parameters:

• timesteps (time steps.) –

• input ((none, timesteps, nb_variables)) –

• output ((none, timesteps, nb_variables)) –

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.FeedForwardAttention(*args, **kwargs)

Bases: Layer

Feed Forward Attention.

Follows the work of Raffel et al. [https://arxiv.org/abs/1512.08756]

input: (none, timesteps, nb_variables) output: (none, nb_variables) or (none, timesteps, nb_variables)

Parameters:

• return_sequences (Boolean. Whether to return the last output. in the output) – sequence, or the full sequence. Default: False.

• use_bias (Boolean. Whether the layer uses a bias vector. Default: False.) –

• activation (Str. Activation function to use. Default: tanh.) –

• Example – model.add(LSTM(64, return_sequences=True)) model.add(FFAttention())

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.AutoRegressive(*args, **kwargs)

Bases: Layer

AutoRegressive Layer based on Dense.

Parameters:

• order (lookback step for original sequence.) –

• input ((none, timesteps, nb_variables)) –

• output ((none, nb_variables)) –

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.Highway(*args, **kwargs)

Bases: Layer

Highway to implement skip conenction based on NIN and GlobalAveragePooling1D.

Parameters:

• input ((none, timesteps, nb_variables)) –

• output ((none, nb_variables)) –

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.Time2Vec(*args, **kwargs)

Bases: Layer

The vector representation for time series.

Parameters:

• kernel_size (int, dimension of vector embedding.) –

• periodic_activation (str, optional {'sin', 'cos'}, default 'sin'.) –

• input ((none, timesteps, nb_variables)) –

• output ((none, timesteps, kernel_size+nb_variables)) –

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.RevInstanceNormalization(*args, **kwargs)

Bases: Layer

Reversible Instance Normalization for Accurate Time-Series Forecasting

against Distribution Shift, ICLR2022.

Parameters:

• eps (float, a value added for numerical stability, default 1e-5.) –

• affine (bool, if True(default), RevIN has learnable affine parameters.) –

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.Identity(*args, **kwargs)

Bases: Layer

Identity Layer.

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.Shortcut(*args, **kwargs)

Bases: Layer

Shortcut Layer.

Parampers

filters: the dimensionality of the output space for Conv1D.

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.InceptionBlock(*args, **kwargs)

Bases: Layer

InceptionBlock for time series.

Parampers

filters: the dimensionality of the output space for Conv1D. kernel_size_list: list or tuple, a list of kernel size for Conv1D. strides: int or tuple, default 1. use_bottleneck: bool, whether to use bottleneck, default True. bottleneck_size: int, if use bottleneck, bottleneck_size is 32(default). activation: str, activation function, default ‘relu’.

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.FactorizedReduce(*args, **kwargs)

Bases: Layer

Factorized reduce for timestemp or variable.

Parampers

period: lookback step for original sequence. filters: the dimensionality of the output space for Conv1D. strides: int or tuple, default 1.

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

class hyperts.framework.dl.layers._layers.Sampling(*args, **kwargs)

Bases: Layer

Reparametrisation by sampling from Gaussian N(0,I).

Parampers

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE, **kwargs)

Adds a new variable to the layer.

Parameters:

• name – Variable name.

• shape – Variable shape. Defaults to scalar if unspecified.

• dtype – The type of the variable. Defaults to self.dtype.

• initializer – Initializer instance (callable).

• regularizer – Regularizer instance (callable).

• trainable – Boolean, whether the variable should be part of the layer’s “trainable_variables” (e.g. variables, biases) or “non_trainable_variables” (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.

• constraint – Constraint instance (callable).

• use_resource – Whether to use ResourceVariable.

• synchronization – Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.

• aggregation – Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.

• **kwargs – Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

Returns:

The variable created.

Raises:

ValueError – When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

hyperts.framework.dl.layers._layers.build_input_head(window, continuous_columns, categorical_columns)

Build the input head. An input variable may have two parts: continuous variables

and categorical variables.

Parameters:

• window (length of the time series sequences for a sample, i.e., timestamp.) –

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

hyperts.framework.dl.layers._layers.build_denses(continuous_columns, continuous_inputs, use_layernormalization=False)

Concatenate continuous inputs.

Parameters:

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• continuous_inputs (list, tf.keras.layers.Input objects.) –

• use_layernormalization (bool, default False.) –

hyperts.framework.dl.layers._layers.build_embeddings(categorical_columns, categorical_inputs)

Build embeddings if there are categorical variables.

Parameters:

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

• categorical_inputs (list, tf.keras.layers.Input objects.) –

hyperts.framework.dl.layers._layers.build_output_tail(x, task, nb_outputs, nb_steps=1)

Build the output tail.

Parameters:

• task (str, See hyperts.utils.consts for details.) –

• nb_outputs (int, the number of output units.) –

• nb_steps (int, the step length of forecast, default 1.) –

hyperts.framework.dl.layers._layers.rnn_forward(x, nb_units, nb_layers, rnn_type, name, drop_rate=0.0, i=0, activation='tanh', return_sequences=False)

Multi-RNN layers.

Parameters:

• nb_units (int, the dimensionality of the output space for) – recurrent neural network.

• nb_layers (int, the number of the layers for recurrent neural network.) –

• rnn_type (str, type of recurrent neural network,) – including {‘basic’, ‘gru’, ‘lstm’}.

• name (recurrent neural network name.) –

• drop_rate (float, the rate of Dropout for neural nets, default 0.) –

• return_sequences (bool, whether to return the last output. in the output) – sequence, or the full sequence. default False.

dl.losses

class hyperts.framework.dl.losses._losses.LogGaussianLoss(name='log_gaussian_loss', **kwargs)

Bases: LossFunctionWrapper

Log Gaussian loss, is applied to DeepAR.

Parameters:

name – (Optional) string name of the metric instance.

Usage with compile() API:

```python model.compile(

optimizer=’sgd’, loss=LogGaussianLoss(), metrics=[‘mse’])

```

call(y_true, y_pred)

Invokes the LossFunctionWrapper instance.

Parameters:

• y_true – Ground truth values.

• y_pred – The predicted values.

Returns:

Loss values per sample.

classmethod from_config(config)

Instantiates a Loss from its config (output of get_config()).

Parameters:

config – Output of get_config().

Returns:

A Loss instance.

get_config()

Returns the config dictionary for a Loss instance.

class hyperts.framework.dl.losses._losses.SymmetricMeanAbsolutePercentageError(name='symmetric_mean_absolute_percentage_loss', **kwargs)

Bases: LossFunctionWrapper

Symmetric Mean Absolute Percentage Error loss.

Parameters:

name – (Optional) string name of the metric instance.

Usage with compile() API:

```python model.compile(

optimizer=’sgd’, loss=SymmetricMeanAbsolutePercentageLoss(), metrics=[‘mse’])

```

call(y_true, y_pred)

Invokes the LossFunctionWrapper instance.

Parameters:

• y_true – Ground truth values.

• y_pred – The predicted values.

Returns:

Loss values per sample.

classmethod from_config(config)

Instantiates a Loss from its config (output of get_config()).

Parameters:

config – Output of get_config().

Returns:

A Loss instance.

get_config()

Returns the config dictionary for a Loss instance.

hyperts.framework.dl.losses._losses.log_gaussian_error(y_true, y_pred)

hyperts.framework.dl.losses._losses.symmetric_mean_absolute_percentage_error(y_true, y_pred)

dl.metrics

class hyperts.framework.dl.metrics._metrics.SymmetricMeanAbsolutePercentageError(*args, **kwargs)

Bases: MeanMetricWrapper

Computes the symmetric mean absolute percentage error between y_true and y_pred.

Parameters:

name – (Optional) string name of the metric instance.

Usage with compile() API:

```python model.compile(

optimizer=’sgd’, loss=’mse’, metrics=[SymmetricMeanAbsolutePercentageError()])

```

property activity_regularizer

Optional regularizer function for the output of this layer.

add_loss(losses, **kwargs)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model’s call function, in which case losses should be a Tensor or list of Tensors.

Example:

```python class MyLayer(tf.keras.layers.Layer):

def call(self, inputs):

self.add_loss(tf.abs(tf.reduce_mean(inputs))) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These losses become part of the model’s topology and are tracked in `get_config.

Example:

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Activity regularization. model.add_loss(tf.abs(tf.reduce_mean(x))) `

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model’s layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model’s topology since they can’t be serialized.

Example:

`python inputs = tf.keras.Input(shape=(10,)) d = tf.keras.layers.Dense(10) x = d(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) # Weight regularization. model.add_loss(lambda: tf.reduce_mean(d.kernel)) `

Parameters:

• losses – Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.

• **kwargs –

  Additional keyword arguments for backward compatibility. Accepted values:

  inputs - Deprecated, will be automatically inferred.

add_metric(value, name=None, **kwargs)

Adds metric tensor to the layer.

This method can be used inside the call() method of a subclassed layer or model.

```python class MyMetricLayer(tf.keras.layers.Layer):

def __init__(self):

super(MyMetricLayer, self).__init__(name=’my_metric_layer’) self.mean = tf.keras.metrics.Mean(name=’metric_1’)

def call(self, inputs):

self.add_metric(self.mean(x)) self.add_metric(tf.reduce_sum(x), name=’metric_2’) return inputs

```

This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model’s Input`s. These metrics become part of the model’s topology and are tracked when you save the model via `save().

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(math_ops.reduce_sum(x), name='metric_1') `

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model’s inputs.

`python inputs = tf.keras.Input(shape=(10,)) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Dense(1)(x) model = tf.keras.Model(inputs, outputs) model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1') `

Parameters:

• value – Metric tensor.

• name – String metric name.

• **kwargs – Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.

add_update(updates, inputs=None)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.updates may be dependent on a and some on b. This method automatically keeps track of dependencies.

This call is ignored when eager execution is enabled (in that case, variable updates are run on the fly and thus do not need to be tracked for later execution).

Parameters:

• updates – Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting trainable=False on this Layer, when executing in Eager mode.

• inputs – Deprecated, will be automatically inferred.

add_variable(*args, **kwargs)

Deprecated, do NOT use! Alias for add_weight.

add_weight(name, shape=(), aggregation=VariableAggregation.SUM, synchronization=VariableSynchronization.ON_READ, initializer=None, dtype=None)

Adds state variable. Only for use by subclasses.

apply(inputs, *args, **kwargs)

Deprecated, do NOT use!

This is an alias of self.__call__.

Parameters:

• inputs – Input tensor(s).

• *args – additional positional arguments to be passed to self.call.

• **kwargs – additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call(inputs, **kwargs)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

• inputs – Input tensor, or list/tuple of input tensors.

• **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

property compute_dtype

The dtype of the layer’s computations.

This is equivalent to Layer.dtype_policy.compute_dtype. Unless mixed precision is used, this is the same as Layer.dtype, the dtype of the weights.

Layers automatically cast their inputs to the compute dtype, which causes computations and the output to be in the compute dtype as well. This is done by the base Layer class in Layer.__call__, so you do not have to insert these casts if implementing your own layer.

Layers often perform certain internal computations in higher precision when compute_dtype is float16 or bfloat16 for numeric stability. The output will still typically be float16 or bfloat16 in such cases.

Returns:

The layer’s compute dtype.

compute_mask(inputs, mask=None)

Computes an output mask tensor.

Parameters:

• inputs – Tensor or list of tensors.

• mask – Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors,

one per output tensor of the layer).

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Parameters:

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

compute_output_signature(input_signature)

Compute the output tensor signature of the layer based on the inputs.

Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn’t implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

Parameters:

input_signature – Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

Returns:

Single TensorSpec or nested structure of TensorSpec objects, describing

how the layer would transform the provided input.

Raises:

TypeError – If input_signature contains a non-TensorSpec object.

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

ValueError – if the layer isn’t yet built (in which case its weights aren’t yet defined).

property dtype

The dtype of the layer weights.

This is equivalent to Layer.dtype_policy.variable_dtype. Unless mixed precision is used, this is the same as Layer.compute_dtype, the dtype of the layer’s computations.

property dtype_policy

The dtype policy associated with this layer.

This is an instance of a tf.keras.mixed_precision.Policy.

property dynamic

Whether the layer is dynamic (eager-only); set in the constructor.

classmethod from_config(config)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

Parameters:

config – A Python dictionary, typically the output of get_config.

Returns:

A layer instance.

get_config()

Returns the serializable config of the metric.

get_input_at(node_index)

Retrieves the input tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_input_mask_at(node_index)

Retrieves the input mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at(node_index)

Retrieves the input shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

RuntimeError – If called in Eager mode.

get_losses_for(inputs)

Deprecated, do NOT use!

Retrieves losses relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at(node_index)

Retrieves the output tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_output_mask_at(node_index)

Retrieves the output mask tensor(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at(node_index)

Retrieves the output shape(s) of a layer at a given node.

Parameters:

node_index – Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

RuntimeError – If called in Eager mode.

get_updates_for(inputs)

Deprecated, do NOT use!

Retrieves updates relevant to a specific set of inputs.

Parameters:

inputs – Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights()

Returns the current weights of the layer.

The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Returns:

Weights values as a list of numpy arrays.

property inbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property input

Retrieves the input tensor(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

Returns:

Input tensor or list of input tensors.

Raises:

• RuntimeError – If called in Eager mode.

• AttributeError – If no inbound nodes are found.

property input_mask

Retrieves the input mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Input mask tensor (potentially None) or list of input mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property input_shape

Retrieves the input shape(s) of a layer.

Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

Returns:

Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor).

Raises:

• AttributeError – if the layer has no defined input_shape.

• RuntimeError – if called in Eager mode.

property input_spec

InputSpec instance(s) describing the input format for this layer.

When you create a layer subclass, you can set self.input_spec to enable the layer to run input compatibility checks when it is called. Consider a Conv2D layer: it can only be called on a single input tensor of rank 4. As such, you can set, in __init__():

`python self.input_spec = tf.keras.layers.InputSpec(ndim=4) `

Now, if you try to call the layer on an input that isn’t rank 4 (for instance, an input of shape (2,), it will raise a nicely-formatted error:

` ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=1. Full shape received: [2] `

Input checks that can be specified via input_spec include: - Structure (e.g. a single input, a list of 2 inputs, etc) - Shape - Rank (ndim) - Dtype

For more information, see tf.keras.layers.InputSpec.

Returns:

A tf.keras.layers.InputSpec instance, or nested structure thereof.

property losses

List of losses added using the add_loss() API.

Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

Examples:

>>> class MyLayer(tf.keras.layers.Layer):
...   def call(self, inputs):
...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
...     return inputs
>>> l = MyLayer()
>>> l(np.ones((10, 1)))
>>> l.losses
[1.0]

>>> inputs = tf.keras.Input(shape=(10,))
>>> x = tf.keras.layers.Dense(10)(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Activity regularization.
>>> len(model.losses)
0
>>> model.add_loss(tf.abs(tf.reduce_mean(x)))
>>> len(model.losses)
1

>>> inputs = tf.keras.Input(shape=(10,))
>>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
>>> x = d(inputs)
>>> outputs = tf.keras.layers.Dense(1)(x)
>>> model = tf.keras.Model(inputs, outputs)
>>> # Weight regularization.
>>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
>>> model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

Returns:

A list of tensors.

property metrics

List of metrics added using the add_metric() API.

Example:

>>> input = tf.keras.layers.Input(shape=(3,))
>>> d = tf.keras.layers.Dense(2)
>>> output = d(input)
>>> d.add_metric(tf.reduce_max(output), name='max')
>>> d.add_metric(tf.reduce_min(output), name='min')
>>> [m.name for m in d.metrics]
['max', 'min']

Returns:

A list of Metric objects.

property name

Name of the layer (string), set in the constructor.

property name_scope

Returns a tf.name_scope instance for this class.

property non_trainable_variables

property non_trainable_weights

List of all non-trainable weights tracked by this layer.

Non-trainable weights are not updated during training. They are expected to be updated manually in call().

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of non-trainable variables.

property outbound_nodes

Deprecated, do NOT use! Only for compatibility with external Keras.

property output

Retrieves the output tensor(s) of a layer.

Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

Returns:

Output tensor or list of output tensors.

Raises:

• AttributeError – if the layer is connected to more than one incoming layers.

• RuntimeError – if called in Eager mode.

property output_mask

Retrieves the output mask tensor(s) of a layer.

Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

Returns:

Output mask tensor (potentially None) or list of output mask tensors.

Raises:

• AttributeError – if the layer is connected to

• more than one incoming layers. –

property output_shape

Retrieves the output shape(s) of a layer.

Only applicable if the layer has one output, or if all outputs have the same shape.

Returns:

Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor).

Raises:

• AttributeError – if the layer has no defined output shape.

• RuntimeError – if called in Eager mode.

reset_states()

Resets all of the metric state variables.

This function is called between epochs/steps, when a metric is evaluated during training.

result()

Computes and returns the metric value tensor.

Result computation is an idempotent operation that simply calculates the metric value using the state variables.

set_weights(weights)

Sets the weights of the layer, from Numpy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer’s weights must be instantiated before calling this function by calling the layer.

For example, a Dense layer returns a list of two values– per-output weights and the bias value. These can be used to set the weights of another Dense layer:

>>> a = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(1.))
>>> a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
>>> a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
>>> b = tf.keras.layers.Dense(1,
...   kernel_initializer=tf.constant_initializer(2.))
>>> b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
>>> b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
>>> b.set_weights(a.get_weights())
>>> b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

Parameters:

weights – a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

ValueError – If the provided weights list does not match the layer’s specifications.

property stateful

property submodules

Sequence of all sub-modules.

Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

>>> a = tf.Module()
>>> b = tf.Module()
>>> c = tf.Module()
>>> a.b = b
>>> b.c = c
>>> list(a.submodules) == [b, c]
True
>>> list(b.submodules) == [c]
True
>>> list(c.submodules) == []
True

Returns:

A sequence of all submodules.

property supports_masking

Whether this layer supports computing a mask using compute_mask.

property trainable

property trainable_variables

Sequence of trainable variables owned by this module and its submodules.

Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don’t expect the return value to change.

Returns:

A sequence of variables for the current module (sorted by attribute name) followed by variables from all submodules recursively (breadth first).

property trainable_weights

List of all trainable weights tracked by this layer.

Trainable weights are updated via gradient descent during training.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of trainable variables.

update_state(y_true, y_pred, sample_weight=None)

Accumulates metric statistics.

y_true and y_pred should have the same shape.

Parameters:

• y_true – Ground truth values. shape = [batch_size, d0, .. dN].

• y_pred – The predicted values. shape = [batch_size, d0, .. dN].

• sample_weight – Optional sample_weight acts as a coefficient for the metric. If a scalar is provided, then the metric is simply scaled by the given value. If sample_weight is a tensor of size [batch_size], then the metric for each sample of the batch is rescaled by the corresponding element in the sample_weight vector. If the shape of sample_weight is [batch_size, d0, .. dN-1] (or can be broadcasted to this shape), then each metric element of y_pred is scaled by the corresponding value of sample_weight. (Note on dN-1: all metric functions reduce by 1 dimension, usually the last axis (-1)).

Returns:

Update op.

property updates

property variable_dtype

Alias of Layer.dtype, the dtype of the weights.

property variables

Returns the list of all layer variables/weights.

Alias of self.weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

property weights

Returns the list of all layer variables/weights.

Note: This will not track the weights of nested tf.Modules that are not themselves Keras layers.

Returns:

A list of variables.

classmethod with_name_scope(method)

Decorator to automatically enter the module name scope.

>>> class MyModule(tf.Module):
...   @tf.Module.with_name_scope
...   def __call__(self, x):
...     if not hasattr(self, 'w'):
...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
...     return tf.matmul(x, self.w)

Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose names included the module name:

>>> mod = MyModule()
>>> mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
>>> mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

Parameters:

method – The method to wrap.

Returns:

The original method wrapped such that it enters the module’s name scope.

dl.optimizers

AdamP for TensorFlow.

class hyperts.framework.dl.optimizers._optimizers.AdamP(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, weight_decay=0.0, delta=0.1, wd_ratio=0.1, nesterov=False, name='AdamP', **kwargs)

Bases: OptimizerV2

Construct a new AdamP optimizer.

Follows the work of Byeongho Heo et al. [https://arxiv.org/abs/2006.08217]

Parameters:

• learning_rate (A Tensor, floating point value, or a schedule that is a) – tf.keras.optimizers.schedules.LearningRateSchedule, or a callable that takes no arguments and returns the actual value to use, The learning rate. Defaults to 0.001.

• beta_1 (A float value or a constant float tensor, or a callable) – that takes no arguments and returns the actual value to use. The exponential decay rate for the 1st moment estimates. Defaults to 0.9.

• beta_2 (A float value or a constant float tensor, or a callable) – that takes no arguments and returns the actual value to use, The exponential decay rate for the 2nd moment estimates. Defaults to 0.999.

• epsilon (A small constant for numerical stability. This epsilon is) – “epsilon hat” in the Kingma and Ba paper (in the formula just before Section 2.1), not the epsilon in Algorithm 1 of the paper. Defaults to 1e-8.

• weight_decay (A Tensor or a floating point value. The weight decay. Defaults to 0.) –

• delta (threhold that determines whether a set of parameters is scale invariant or) – not. Defaults to 0.1.

• wd_ratio (relative weight decay applied on scale-invariant parameters compared to) – that applied on scale-variant parameters. Defaults to 0.1.

• name (Optional name for the operations created when applying gradients.) – Defaults to “AdamP”.

• **kwargs (Keyword arguments. Allowed to be one of) – “clipnorm” or “clipvalue”. “clipnorm” (float) clips gradients by norm; “clipvalue” (float) clips gradients by value.

References

https://github.com/taki0112/AdamP-Tensorflow

set_weights(weights)

Set the weights of the optimizer.

The weights of an optimizer are its state (ie, variables). This function takes the weight values associated with this optimizer as a list of Numpy arrays. The first value is always the iterations count of the optimizer, followed by the optimizer’s state variables in the order they are created. The passed values are used to set the new state of the optimizer.

For example, the RMSprop optimizer for this simple model takes a list of three values– the iteration count, followed by the root-mean-square value of the kernel and bias of the single Dense layer:

>>> opt = tf.keras.optimizers.RMSprop()
>>> m = tf.keras.models.Sequential([tf.keras.layers.Dense(10)])
>>> m.compile(opt, loss='mse')
>>> data = np.arange(100).reshape(5, 20)
>>> labels = np.zeros(5)
>>> print('Training'); results = m.fit(data, labels)
Training ...
>>> new_weights = [np.array(10), np.ones([20, 10]), np.zeros([10])]
>>> opt.set_weights(new_weights)
>>> opt.iterations
<tf.Variable 'RMSprop/iter:0' shape=() dtype=int64, numpy=10>

Parameters:

weights – weight values as a list of numpy arrays.

channel_true_fn(var, perturb, wd_ratio, eps)

channel_false_fn(var, grad, perturb, delta, wd_ratio, eps)

layer_true_fn(var, perturb, wd_ratio, eps)

identity_fn(perturb)

get_config()

Returns the config of the optimizer.

An optimizer config is a Python dictionary (serializable) containing the configuration of an optimizer. The same optimizer can be reinstantiated later (without any saved state) from this configuration.

Returns:

Python dictionary.

add_slot(var, slot_name, initializer='zeros')

Add a new slot variable for var.

add_weight(name, shape, dtype=None, initializer='zeros', trainable=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE)

apply_gradients(grads_and_vars, name=None, experimental_aggregate_gradients=True)

Apply gradients to variables.

This is the second part of minimize(). It returns an Operation that applies gradients.

The method sums gradients from all replicas in the presence of tf.distribute.Strategy by default. You can aggregate gradients yourself by passing experimental_aggregate_gradients=False.

Example:

```python grads = tape.gradient(loss, vars) grads = tf.distribute.get_replica_context().all_reduce(‘sum’, grads) # Processing aggregated gradients. optimizer.apply_gradients(zip(grads, vars),

experimental_aggregate_gradients=False)

```

Parameters:

• grads_and_vars – List of (gradient, variable) pairs.

• name – Optional name for the returned operation. Default to the name passed to the Optimizer constructor.

• experimental_aggregate_gradients – Whether to sum gradients from different replicas in the presense of tf.distribute.Strategy. If False, it’s user responsibility to aggregate the gradients. Default to True.

Returns:

An Operation that applies the specified gradients. The iterations will be automatically increased by 1.

Raises:

• TypeError – If grads_and_vars is malformed.

• ValueError – If none of the variables have gradients.

• RuntimeError – If called in a cross-replica context.

property clipnorm

float or None. If set, clips gradients to a maximum norm.

property clipvalue

float or None. If set, clips gradients to a maximum value.

classmethod from_config(config, custom_objects=None)

Creates an optimizer from its config.

This method is the reverse of get_config, capable of instantiating the same optimizer from the config dictionary.

Parameters:

• config – A Python dictionary, typically the output of get_config.

• custom_objects – A Python dictionary mapping names to additional Python objects used to create this optimizer, such as a function used for a hyperparameter.

Returns:

An optimizer instance.

get_gradients(loss, params)

Returns gradients of loss with respect to params.

Should be used only in legacy v1 graph mode.

Parameters:

• loss – Loss tensor.

• params – List of variables.

Returns:

List of gradient tensors.

Raises:

ValueError – In case any gradient cannot be computed (e.g. if gradient function not implemented).

get_slot(var, slot_name)

get_slot_names()

A list of names for this optimizer’s slots.

get_updates(loss, params)

get_weights()

Returns the current weights of the optimizer.

The weights of an optimizer are its state (ie, variables). This function returns the weight values associated with this optimizer as a list of Numpy arrays. The first value is always the iterations count of the optimizer, followed by the optimizer’s state variables in the order they were created. The returned list can in turn be used to load state into similarly parameterized optimizers.

For example, the RMSprop optimizer for this simple model returns a list of three values– the iteration count, followed by the root-mean-square value of the kernel and bias of the single Dense layer:

>>> opt = tf.keras.optimizers.RMSprop()
>>> m = tf.keras.models.Sequential([tf.keras.layers.Dense(10)])
>>> m.compile(opt, loss='mse')
>>> data = np.arange(100).reshape(5, 20)
>>> labels = np.zeros(5)
>>> print('Training'); results = m.fit(data, labels)
Training ...
>>> len(opt.get_weights())
3

Returns:

Weights values as a list of numpy arrays.

property global_clipnorm

float or None. If set, clips gradients to a maximum norm.

property iterations

Variable. The number of training steps this Optimizer has run.

minimize(loss, var_list, grad_loss=None, name=None, tape=None)

Minimize loss by updating var_list.

This method simply computes gradient using tf.GradientTape and calls apply_gradients(). If you want to process the gradient before applying then call tf.GradientTape and apply_gradients() explicitly instead of using this function.

Parameters:

• loss – Tensor or callable. If a callable, loss should take no arguments and return the value to minimize. If a Tensor, the tape argument must be passed.

• var_list – list or tuple of Variable objects to update to minimize loss, or a callable returning the list or tuple of Variable objects. Use callable when the variable list would otherwise be incomplete before minimize since the variables are created at the first time loss is called.

• grad_loss – (Optional). A Tensor holding the gradient computed for loss.

• name – (Optional) str. Name for the returned operation.

• tape – (Optional) tf.GradientTape. If loss is provided as a Tensor, the tape that computed the loss must be provided.

Returns:

An Operation that updates the variables in var_list. The iterations will be automatically increased by 1.

Raises:

ValueError – If some of the variables are not Variable objects.

variables()

Returns variables of this Optimizer based on the order created.

property weights

Returns variables of this Optimizer based on the order created.

class hyperts.framework.dl.optimizers._optimizers.Lion(learning_rate=0.0001, beta_1=0.9, beta_2=0.99, wd_ratio=0, name='lion', **kwargs)

Bases: OptimizerV2

Optimizer that implements the Lion algorithm.

Parameters:

• params (iterable) – iterable of parameters to optimize or dicts defining parameter groups

• lr (float, optional) – learning rate (default: 1e-4)

• betas (Tuple[float, float], optional) – coefficients used for computing running averages of gradient and its square (default: (0.9, 0.99))

• weight_decay (float, optional) – weight decay coefficient (default: 0)

References

https://github.com/google/automl/blob/master/lion/lion_tf2.py

get_config()

Returns the config of the optimizer.

An optimizer config is a Python dictionary (serializable) containing the configuration of an optimizer. The same optimizer can be reinstantiated later (without any saved state) from this configuration.

Returns:

Python dictionary.

add_slot(var, slot_name, initializer='zeros')

Add a new slot variable for var.

add_weight(name, shape, dtype=None, initializer='zeros', trainable=None, synchronization=VariableSynchronization.AUTO, aggregation=VariableAggregation.NONE)

apply_gradients(grads_and_vars, name=None, experimental_aggregate_gradients=True)

Apply gradients to variables.

This is the second part of minimize(). It returns an Operation that applies gradients.

The method sums gradients from all replicas in the presence of tf.distribute.Strategy by default. You can aggregate gradients yourself by passing experimental_aggregate_gradients=False.

Example:

```python grads = tape.gradient(loss, vars) grads = tf.distribute.get_replica_context().all_reduce(‘sum’, grads) # Processing aggregated gradients. optimizer.apply_gradients(zip(grads, vars),

experimental_aggregate_gradients=False)

```

Parameters:

• grads_and_vars – List of (gradient, variable) pairs.

• name – Optional name for the returned operation. Default to the name passed to the Optimizer constructor.

• experimental_aggregate_gradients – Whether to sum gradients from different replicas in the presense of tf.distribute.Strategy. If False, it’s user responsibility to aggregate the gradients. Default to True.

Returns:

An Operation that applies the specified gradients. The iterations will be automatically increased by 1.

Raises:

• TypeError – If grads_and_vars is malformed.

• ValueError – If none of the variables have gradients.

• RuntimeError – If called in a cross-replica context.

property clipnorm

float or None. If set, clips gradients to a maximum norm.

property clipvalue

float or None. If set, clips gradients to a maximum value.

classmethod from_config(config, custom_objects=None)

Creates an optimizer from its config.

This method is the reverse of get_config, capable of instantiating the same optimizer from the config dictionary.

Parameters:

• config – A Python dictionary, typically the output of get_config.

• custom_objects – A Python dictionary mapping names to additional Python objects used to create this optimizer, such as a function used for a hyperparameter.

Returns:

An optimizer instance.

get_gradients(loss, params)

Returns gradients of loss with respect to params.

Should be used only in legacy v1 graph mode.

Parameters:

• loss – Loss tensor.

• params – List of variables.

Returns:

List of gradient tensors.

Raises:

ValueError – In case any gradient cannot be computed (e.g. if gradient function not implemented).

get_slot(var, slot_name)

get_slot_names()

A list of names for this optimizer’s slots.

get_updates(loss, params)

get_weights()

Returns the current weights of the optimizer.

The weights of an optimizer are its state (ie, variables). This function returns the weight values associated with this optimizer as a list of Numpy arrays. The first value is always the iterations count of the optimizer, followed by the optimizer’s state variables in the order they were created. The returned list can in turn be used to load state into similarly parameterized optimizers.

For example, the RMSprop optimizer for this simple model returns a list of three values– the iteration count, followed by the root-mean-square value of the kernel and bias of the single Dense layer:

>>> opt = tf.keras.optimizers.RMSprop()
>>> m = tf.keras.models.Sequential([tf.keras.layers.Dense(10)])
>>> m.compile(opt, loss='mse')
>>> data = np.arange(100).reshape(5, 20)
>>> labels = np.zeros(5)
>>> print('Training'); results = m.fit(data, labels)
Training ...
>>> len(opt.get_weights())
3

Returns:

Weights values as a list of numpy arrays.

property global_clipnorm

float or None. If set, clips gradients to a maximum norm.

property iterations

Variable. The number of training steps this Optimizer has run.

minimize(loss, var_list, grad_loss=None, name=None, tape=None)

Minimize loss by updating var_list.

This method simply computes gradient using tf.GradientTape and calls apply_gradients(). If you want to process the gradient before applying then call tf.GradientTape and apply_gradients() explicitly instead of using this function.

Parameters:

• loss – Tensor or callable. If a callable, loss should take no arguments and return the value to minimize. If a Tensor, the tape argument must be passed.

• var_list – list or tuple of Variable objects to update to minimize loss, or a callable returning the list or tuple of Variable objects. Use callable when the variable list would otherwise be incomplete before minimize since the variables are created at the first time loss is called.

• grad_loss – (Optional). A Tensor holding the gradient computed for loss.

• name – (Optional) str. Name for the returned operation.

• tape – (Optional) tf.GradientTape. If loss is provided as a Tensor, the tape that computed the loss must be provided.

Returns:

An Operation that updates the variables in var_list. The iterations will be automatically increased by 1.

Raises:

ValueError – If some of the variables are not Variable objects.

set_weights(weights)

Set the weights of the optimizer.

The weights of an optimizer are its state (ie, variables). This function takes the weight values associated with this optimizer as a list of Numpy arrays. The first value is always the iterations count of the optimizer, followed by the optimizer’s state variables in the order they are created. The passed values are used to set the new state of the optimizer.

For example, the RMSprop optimizer for this simple model takes a list of three values– the iteration count, followed by the root-mean-square value of the kernel and bias of the single Dense layer:

>>> opt = tf.keras.optimizers.RMSprop()
>>> m = tf.keras.models.Sequential([tf.keras.layers.Dense(10)])
>>> m.compile(opt, loss='mse')
>>> data = np.arange(100).reshape(5, 20)
>>> labels = np.zeros(5)
>>> print('Training'); results = m.fit(data, labels)
Training ...
>>> new_weights = [np.array(10), np.ones([20, 10]), np.zeros([10])]
>>> opt.set_weights(new_weights)
>>> opt.iterations
<tf.Variable 'RMSprop/iter:0' shape=() dtype=int64, numpy=10>

Parameters:

weights – weight values as a list of numpy arrays.

variables()

Returns variables of this Optimizer based on the order created.

property weights

Returns variables of this Optimizer based on the order created.

dl.dl_utils.metainfo

class hyperts.framework.dl.dl_utils.metainfo.CategoricalColumn(name, vocabulary_size, embedding_dim=4, dtype='int32', input_name=None)

Bases: CategoricalColumn

count(value, /)

Return number of occurrences of value.

property dtype

Alias for field number 3

property embedding_dim

Alias for field number 2

index(value, start=0, stop=9223372036854775807, /)

Return first index of value.

Raises ValueError if the value is not present.

property input_name

Alias for field number 4

property name

Alias for field number 0

property vocabulary_size

Alias for field number 1

class hyperts.framework.dl.dl_utils.metainfo.ContinuousColumn(name, column_names, input_dim=0, dtype='float32', input_name=None)

Bases: CotinuousColumn

property column_names

Alias for field number 1

count(value, /)

Return number of occurrences of value.

property dtype

Alias for field number 3

index(value, start=0, stop=9223372036854775807, /)

Return first index of value.

Raises ValueError if the value is not present.

property input_dim

Alias for field number 2

property input_name

Alias for field number 4

property name

Alias for field number 0

class hyperts.framework.dl.dl_utils.metainfo.MetaPreprocessor

Bases: object

Abstract base class representing Meta Preprocessor.

property pos_label

property labels

property transformers

fit_transform(X, y, copy_data=True)

transform_X(X, copy_data=True)

transform_y(y, copy_data=True)

transform(X, y, copy_data=True)

inverse_transform_y(y_indicator)

get_categorical_columns()

get_continuous_columns()

class hyperts.framework.dl.dl_utils.metainfo.MetaTSFprocessor(timestamp, embedding_output_dim=4, auto_categorize=False, auto_encode_label=True, cat_remain_numeric=True)

Bases: MetaPreprocessor

Mata Time Series Forecast Processor.

Parameters:

• timestamp (str, time column name (DataFrame).) –

• embedding_output_dim (int, default 4.) – Embed dimension when there are categorical variables.

• auto_categorize (bool, default False.) –

• auto_encode_label (bool, default True.) –

• cat_remain_numeric (bool, default True.) –

transform_X(X, copy_data=True)

Transform X.

fit_transform(X, y, copy_data=True)

Fit and Transform.

transform(X, y, copy_data=True)

Transform.

get_categorical_columns()

get_continuous_columns()

inverse_transform_y(y_indicator)

property labels

property pos_label

transform_y(y, copy_data=True)

property transformers

class hyperts.framework.dl.dl_utils.metainfo.MetaTSCprocessor(task, embedding_output_dim=4, auto_categorize=False, auto_discard_unique=True, cat_remain_numeric=True)

Bases: MetaPreprocessor

Mata Time Series Classification or Regression Processor.

Parameters:

• embedding_output_dim (int, default 4.) – Embed dimension when there are categorical variables.

• auto_categorize (bool, default False.) –

• auto_encode_label (bool, default True.) –

• cat_remain_numeric (bool, default True.) –

transform_X(X, copy_data=False)

Transform X.

fit_transform_y(y)

Fit and Transform y. Transform ont hot encoding for multiclass.

transform_y(y, copy_data=False)

Transform y. Transform ont hot encoding for multiclass.

inverse_transform_y(y_indicator)

Inverse origonal target format.

fit_transform(X, y, copy_data=True)

Fit and Transform.

transform(X, y, copy_data=True)

Transform.

get_categorical_columns()

get_continuous_columns()

property labels

property pos_label

property transformers

dl.dl_utils.timeseries

Keras timeseries dataset utilities.

hyperts.framework.dl.dl_utils.timeseries.from_array_to_timeseries(data, targets, sequence_length, forecast_length=1, sequence_stride=1, sampling_rate=1, batch_size=128, shuffle=False, seed=None, start_index=None, end_index=None)

Creates a dataset of sliding windows over a timeseries provided as array.

This function takes in a sequence of data-points gathered at equal intervals, along with time series parameters such as length of the sequences/windows, spacing between two sequence/windows, etc., to produce batches of timeseries inputs and targets.

Parameters:

• data – Numpy array or eager tensor containing consecutive data points (timesteps). Axis 0 is expected to be the time dimension.

• targets – Targets corresponding to timesteps in data. It should have same length as data. targets[i] should be the target corresponding to the window that starts at index i (see example 2 below). Pass None if you don’t have target data (in this case the dataset will only yield the input data).

• sequence_length – Length of the output sequences (in number of timesteps).

• forecast_length – Step of the forecast outputs.

• sequence_stride – Period between successive output sequences. For stride s, output samples would start at index data[i], data[i + s], data[i + 2 * s], etc.

• sampling_rate – Period between successive individual timesteps within sequences. For rate r, timesteps data[i], data[i + r], … data[i + sequence_length] are used for create a sample sequence.

• batch_size – Number of timeseries samples in each batch (except maybe the last one).

• shuffle – Whether to shuffle output samples, or instead draw them in chronological order.

• seed – Optional int; random seed for shuffling.

• start_index – Optional int; data points earlier (exclusive) than start_index will not be used in the output sequences. This is useful to reserve part of the data for test or validation.

• end_index – Optional int; data points later (exclusive) than end_index will not be used in the output sequences. This is useful to reserve part of the data for test or validation.

Returns:

A tf.data.Dataset instance. If targets was passed, the dataset yields tuple (batch_of_sequences, batch_of_targets). If not, the dataset yields only batch_of_sequences.

Example 1:

Consider indices [0, 1, … 99]. With sequence_length=10, sampling_rate=2, sequence_stride=3, shuffle=False, the dataset will yield batches of sequences composed of the following indices:

` First sequence:  [0  2  4  6  8 10 12 14 16 18] Second sequence: [3  5  7  9 11 13 15 17 19 21] Third sequence:  [6  8 10 12 14 16 18 20 22 24] ... Last sequence:   [78 80 82 84 86 88 90 92 94 96] `

In this case the last 3 data points are discarded since no full sequence can be generated to include them (the next sequence would have started at index 81, and thus its last step would have gone over 99).

Example 2: temporal regression. Consider an array data of scalar values, of shape (steps,). To generate a dataset that uses the past 10 timesteps to predict the next timestep, you would use:

```python input_data = data[:-10] targets = data[10:] dataset = tf.keras.preprocessing.timeseries_dataset_from_array(

input_data, targets, sequence_length=10)

for batch in dataset:

inputs, targets = batch assert np.array_equal(inputs[0], data[:10]) # First sequence: steps [0-9] assert np.array_equal(targets[0], data[10]) # Corresponding target: step 10 break

```

hyperts.framework.dl.dl_utils.timeseries.sequences_from_indices(array, indices_ds, start_index, end_index)

framework.nas

nas.base

class hyperts.framework.nas._base.TSNASEstimator(task, out_activation='linear', timestamp=None, window=3, horizon=1, forecast_length=1, metrics='auto', monitor_metric='val_loss', optimizer='auto', learning_rate=0.001, loss='auto', reducelr_patience=5, earlystop_patience=10, summary=True, continuous_columns=None, categorical_columns=None, **kwargs)

Bases: BaseDeepEstimator

TS-NAS Estimator.

Parameters:

• task (Str - Support forecast, classification, and regression.) – See hyperts.utils.consts for details.

• out_activation (Str - Forecast the task output activation function, optional {'linear', 'sigmoid', 'tanh'},) – default = ‘linear’.

• timestamp (Str or None - Timestamp name, the forecast task must be given,) – default None.

• window (Positive Int - Length of the time series sequences for a sample,) – default = 3.

• horizon (Positive Int - Length of the prediction horizon,) – default = 1.

• forecast_length (Positive Int - Step of the forecast outputs,) – default = 1.

• metrics (Str - List of metrics to be evaluated by the model during training and testing,) – default = ‘auto’.

• monitor_metric (Str - Quality indicators monitored during neural network training.) – default = ‘val_loss’.

• optimizer (Str or keras Instance - for example, 'adam', 'sgd', and so on.) – default = ‘auto’.

• learning_rate (Positive Float - The optimizer's learning rate,) – default = 0.001.

• loss (Str - Loss function, for forecsting or regression, optional {'auto', 'mae', 'mse', 'huber_loss',) – ‘mape’}, for classification, optional {‘auto’, ‘categorical_crossentropy’, ‘binary_crossentropy}, default = ‘auto’.

• reducelr_patience (Positive Int - The number of epochs with no improvement after which learning rate) – will be reduced, default = 5.

• earlystop_patience (Positive Int - The number of epochs with no improvement after which training) – will be stopped, default = 5.

• summary (Bool - Whether to output network structure information,) – default = True.

• continuous_columns (CategoricalColumn class.) – Contains some information(name, column_names, input_dim, dtype, input_name) about continuous variables.

• categorical_columns (CategoricalColumn class.) – Contains some information(name, vocabulary_size, embedding_dim, dtype, input_name) about categorical variables.

build_inputs()

property first_metric_name

Get first metric name.

fit(X, y, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.2, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1, max_queue_size=10, workers=1, use_multiprocessing=False, **kwargs)

Trains the model for a fixed number of epochs (iterations on a dataset).

Parameters:

• X (2D DataFrame of shape: (series_length, 1+(n_covariates)) for forecast task,) – 2D nested DataFrame shape: (sample, nb_covariables(series_length)) for classification or regression task, 2D DataFrame of shape (n_samples, 1+(n_covariates)) for anomaly detection.

• y (2D DataFrame of shape: (series_length, n_target_variables) for forecast task,) – 2D DataFrame of shape: (nb_samples, 1) for classification or regression task, 2D DataFrame of shape (n_samples, reconstract_dim).

• batch_size (Integer or None.) – Number of samples per gradient update. If unspecified, batch_size will default to self-adaption based on number of samples. Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

• epochs (Integer. Number of epochs to train the model.) – An epoch is an iteration over the entire x and y data provided. Note that in conjunction with initial_epoch, epochs is to be understood as “final epoch”. The model is not trained for a number of iterations given by epochs, but merely until the epoch of index epochs is reached.

• verbose ('auto', 0, 1, or 2. Verbosity mode.) – 0 = silent, 1 = progress bar, 2 = one line per epoch. ‘auto’ defaults to 1 for most cases, but 2 when used with ParameterServerStrategy. Note that the progress bar is not particularly useful when logged to a file, so verbose=2 is recommended when not running interactively (eg, in a production environment).

• callbacks (List of keras.callbacks.Callback instances.) – List of callbacks to apply during training. See tf.keras.callbacks. Note tf.keras.callbacks.ProgbarLogger and tf.keras.callbacks.History callbacks are created automatically and need not be passed into model.fit. tf.keras.callbacks.ProgbarLogger is created or not based on verbose argument to model.fit. Callbacks with batch-level calls are currently unsupported with tf.distribute.experimental.ParameterServerStrategy, and users are advised to implement epoch-level calls instead with an appropriate steps_per_epoch value.

• validation_split (Float between 0 and 1.) –

  Fraction of the training data to be used as validation data.

  The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling. This argument is not supported when x is a dataset, generator or

  keras.utils.Sequence instance.

  validation_split is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• validation_data (Data on which to evaluate) –

  the loss and any model metrics at the end of each epoch. The model will not be trained on this data. Thus, note the fact that the validation loss of data provided using validation_split or validation_data is not affected by regularization layers like noise and dropout. validation_data will override validation_split. validation_data could be:

  • A tuple (x_val, y_val) of Numpy arrays or tensors.

  • A tuple (x_val, y_val, val_sample_weights) of NumPy arrays.

  • A tf.data.Dataset.

  • A Python generator or keras.utils.Sequence returning

  (inputs, targets) or (inputs, targets, sample_weights).

  validation_data is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

• shuffle (Boolean (whether to shuffle the training data) – before each epoch) or str (for ‘batch’). This argument is ignored when x is a generator or an object of tf.data.Dataset. ‘batch’ is a special option for dealing with the limitations of HDF5 data; it shuffles in batch-sized chunks. Has no effect when steps_per_epoch is not None.

• class_weight (Optional dictionary mapping class indices (integers)) – to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to “pay more attention” to samples from an under-represented class.

• sample_weight (Optional Numpy array of weights for) –

  the training samples, used for weighting the loss function

  (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. This argument is not supported when x is a dataset, generator, or

  keras.utils.Sequence instance, instead provide the sample_weights

  as the third element of x.

• initial_epoch (Integer.) – Epoch at which to start training (useful for resuming a previous training run).

• steps_per_epoch (Integer or None.) –

  Total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. When training with input tensors such as TensorFlow data tensors, the default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. If x is a tf.data dataset, and ‘steps_per_epoch’ is None, the epoch will run until the input dataset is exhausted. When passing an infinitely repeating dataset, you must specify the steps_per_epoch argument. If steps_per_epoch=-1 the training will run indefinitely with an infinitely repeating dataset. This argument is not supported with array inputs. When using tf.distribute.experimental.ParameterServerStrategy:

  • steps_per_epoch=None is not supported.

• validation_steps (Only relevant if validation_data is provided and) – is a tf.data dataset. Total number of steps (batches of samples) to draw before stopping when performing validation at the end of every epoch. If ‘validation_steps’ is None, validation will run until the validation_data dataset is exhausted. In the case of an infinitely repeated dataset, it will run into an infinite loop. If ‘validation_steps’ is specified and only part of the dataset will be consumed, the evaluation will start from the beginning of the dataset at each epoch. This ensures that the same validation samples are used every time.

• validation_freq (Only relevant if validation data is provided. Integer) – or collections.abc.Container instance (e.g. list, tuple, etc.). If an integer, specifies how many training epochs to run before a new validation run is performed, e.g. validation_freq=2 runs validation every 2 epochs. If a Container, specifies the epochs on which to run validation, e.g. validation_freq=[1, 2, 10] runs validation at the end of the 1st, 2nd, and 10th epochs.

• max_queue_size (Integer. Used for generator or keras.utils.Sequence) – input only. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

• workers (Integer. Used for generator or keras.utils.Sequence input) – only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1.

• use_multiprocessing (Boolean. Used for generator or) – keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes.

• details. (See tf.keras.model.Model.fit for) –

forecast(X)

Inference Function.

Task: time series forecast.

static load_model(model_file, name='dl_model', external=False)

Load the instance object.

property monitor

Gets monitor for ReduceLROnPlateau and EarlyStopping.

plot_net(model_file)

Plot net model architecture.

predict(X, batch_size=128)

Inference Function.

Task: time series classification or regression.

predict_proba(X, batch_size=128)

Inference Function.

Task: time series classification/regression.

proba2predict(proba, encode_to_label=True)

Transition Function.

Task: time series classification.

save_model(model_file, name='dl_model', external=False)

Save the instance object.

nas.layers

hyperts.framework.nas.layers._layers.compile_layer(search_space, layer_class, name, **kwargs)

class hyperts.framework.nas.layers._layers.HyperLayer(keras_layer_class, space=None, name=None, **hyperparams)

Bases: ModuleSpace

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.nas.layers._layers.CalibrateSize(node, name_prefix, space=None, name=None, **hyperparams)

Bases: ModuleSpace

factorized_reduce(name_posfix, period, filters, strides=1)

get_timestemp(x)

get_channels(x)

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

class hyperts.framework.nas.layers._layers.SafeMerge(name_prefix, ops='add', space=None, name=None, **hyperparams)

Bases: ModuleSpace

add_parameters(**hyperparameters)

property all_assigned

attach_to_scope(scope, name=None)

attach_to_space(space=None, name=None)

compile()

compile_and_forward(inputs=None)

connect(module_or_list)

forward(inputs=None)

get_all_params()

get_assignable_params()

property hyper_params

property id

property is_compiled

property is_params_ready

property output

property param_values

property space

property type

update()

hyperts.framework.nas.layers._layers.stem_ops(input, units=64)

hyperts.framework.nas.layers._layers.cell_ops(inputs, name_prefix, block_no, node_no, cell_no, filters_or_units, kernel_size=(1, 3, 5))

hyperts.framework.nas.layers._layers.node_ops(inputs, name_prefix, block_no, node_no, filters_or_units=(16, 32, 64), kernel_size=(1, 3, 5))

hyperts.framework.nas.layers._layers.merge_ops(inputs, name_prefix, ops='add')

class hyperts.framework.nas.layers._layers.LayerWeightsCache

Bases: object

reset()

clear()

hit()

miss()

put(key, layer)

retrieve(key)