report¶
Experiment-tracking backends and pipeline callbacks. For configuration and resume behavior, see Experiment tracking.
mlkit.util.report
¶
NullReporter
¶
Bases: Reporter
Reporter that discards metrics and does not open a run.
Source code in src/mlkit/util/report/base.py
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | |
Reporter
¶
Bases: ABC
Experiment-tracking backend (MLflow, wandb, or none).
Source code in src/mlkit/util/report/base.py
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | |
open()
abstractmethod
¶
Open or attach to a tracking run.
Source code in src/mlkit/util/report/base.py
19 20 21 | |
close(*, success)
abstractmethod
¶
Close the run when this reporter owns it.
Source code in src/mlkit/util/report/base.py
23 24 25 | |
log(data, *, step=None)
abstractmethod
¶
Record one metric payload.
Source code in src/mlkit/util/report/base.py
27 28 29 | |
store_config(config)
abstractmethod
¶
Replace the stored config for the run.
Source code in src/mlkit/util/report/base.py
31 32 33 | |
NullReportCallback
¶
Bases: ReportCallback
Pipeline callback backed by :class:NullReporter.
Source code in src/mlkit/util/report/callback.py
57 58 59 60 61 | |
ReportCallback
¶
Bases: BaseCallback
Forwards pipeline events to a :class:Reporter.
Source code in src/mlkit/util/report/callback.py
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | |
ReporterContext
¶
Class context manager: open on enter, close on exit.
Source code in src/mlkit/util/report/context.py
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | |
MLflowReporter
¶
Bases: Reporter
MLflow tracking backend.
Source code in src/mlkit/util/report/mlflow.py
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 | |
WandbReporter
¶
Bases: Reporter
wandb tracking backend.
Source code in src/mlkit/util/report/wandb.py
24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | |
resume_run_id(checkpoint_dir, backend)
¶
Return the run id to continue, when the checkpoint was written by backend.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
checkpoint_dir
|
str | None
|
latest checkpoint directory, or |
required |
backend
|
str
|
reporter that is about to be opened. |
required |
Returns:
| Type | Description |
|---|---|
str | None
|
The stored run id when the file exists and names the same backend. |
Source code in src/mlkit/util/report/run_file.py
47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | |
write_report_run(directory, *, backend, run_id)
¶
Store the backend run id inside a checkpoint directory.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
directory
|
str
|
checkpoint directory. |
required |
backend
|
str
|
reporter name, |
required |
run_id
|
str
|
id of the run that was open when the checkpoint was saved. |
required |
Source code in src/mlkit/util/report/run_file.py
7 8 9 10 11 12 13 14 15 16 17 | |
get_reporter(kind, *, run_name, config, mlflow=None, wandb=None, resume_run_id=None)
¶
Build a reporter for the given kind. The run is not open until open().
Source code in src/mlkit/util/report/__init__.py
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | |