.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tutorials/metrics/plot_4_curation.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_tutorials_metrics_plot_4_curation.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_tutorials_metrics_plot_4_curation.py:


Curation Tutorial
==================

After spike sorting and computing quality metrics, you can automatically curate the spike sorting output using the
quality metrics that you have calculated.

.. GENERATED FROM PYTHON SOURCE LINES 11-12

Import the modules and/or functions necessary from spikeinterface

.. GENERATED FROM PYTHON SOURCE LINES 12-16

.. code-block:: Python


    import spikeinterface.core as si









.. GENERATED FROM PYTHON SOURCE LINES 17-19

Let's generate a simulated dataset, and imagine that the ground-truth
sorting is in fact the output of a sorter.

.. GENERATED FROM PYTHON SOURCE LINES 19-24

.. code-block:: Python


    recording, sorting = si.generate_ground_truth_recording()
    print(recording)
    print(sorting)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    GroundTruthRecording (InjectTemplatesRecording): 4 channels - 25.0kHz - 1 segments 
                          250,000 samples - 10.00s - float32 dtype - 3.81 MiB
    GroundTruthSorting (NumpySorting): 10 units - 1 segments - 25.0kHz




.. GENERATED FROM PYTHON SOURCE LINES 25-30

Create SortingAnalyzer
-----------------------

For this example, we will need a :code:`SortingAnalyzer` and some extensions
to be computed first

.. GENERATED FROM PYTHON SOURCE LINES 30-39

.. code-block:: Python



    analyzer = si.create_sorting_analyzer(sorting=sorting, recording=recording, format="memory")
    analyzer.compute(["random_spikes", "waveforms", "templates", "noise_levels"])

    analyzer.compute("principal_components", n_components=3, mode="by_channel_local")
    print(analyzer)






.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    estimate_sparsity (no parallelization):   0%|          | 0/10 [00:00<?, ?it/s]
    estimate_sparsity (no parallelization): 100%|██████████| 10/10 [00:00<00:00, 396.40it/s]

    compute_waveforms (no parallelization):   0%|          | 0/10 [00:00<?, ?it/s]
    compute_waveforms (no parallelization): 100%|██████████| 10/10 [00:00<00:00, 303.25it/s]

    noise_level (no parallelization):   0%|          | 0/20 [00:00<?, ?it/s]
    noise_level (no parallelization): 100%|██████████| 20/20 [00:00<00:00, 299.68it/s]

    Fitting PCA:   0%|          | 0/10 [00:00<?, ?it/s]
    Fitting PCA: 100%|██████████| 10/10 [00:00<00:00, 185.98it/s]

    Projecting waveforms:   0%|          | 0/10 [00:00<?, ?it/s]
    Projecting waveforms: 100%|██████████| 10/10 [00:00<00:00, 2625.21it/s]
    SortingAnalyzer: 4 channels - 10 units - 1 segments - memory - sparse - has recording
    Loaded 5 extensions: random_spikes, waveforms, templates, noise_levels, principal_components




.. GENERATED FROM PYTHON SOURCE LINES 40-41

Then we compute some quality metrics:

.. GENERATED FROM PYTHON SOURCE LINES 41-46

.. code-block:: Python


    metrics_ext = analyzer.compute("quality_metrics", metric_names=["snr", "isi_violation", "nearest_neighbor"])
    metrics = metrics_ext.get_data()
    print(metrics)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

             snr  isi_violations_ratio  ...  nn_hit_rate  nn_miss_rate
    0   2.508025                   0.0  ...     0.726241      0.022744
    1  49.534542                   0.0  ...     0.972671      0.005361
    2  21.778729                   0.0  ...     0.839161      0.016458
    3  18.803768                   0.0  ...     0.876129      0.009785
    4  35.940708                   0.0  ...     0.959441      0.002645
    5   3.917549                   0.0  ...     0.777778      0.023521
    6  28.423429                   0.0  ...     0.872368      0.010207
    7  19.725972                   0.0  ...     0.854878      0.021493
    8  12.357215                   0.0  ...     0.842938      0.023662
    9  11.246847                   0.0  ...     0.830075      0.022903

    [10 rows x 5 columns]




.. GENERATED FROM PYTHON SOURCE LINES 47-52

We can now threshold each quality metric and select units based on some rules.

The easiest and most intuitive way is to use boolean masking with a dataframe.

Then create a list of unit ids that we want to keep

.. GENERATED FROM PYTHON SOURCE LINES 52-60

.. code-block:: Python


    keep_mask = (metrics["snr"] > 7.5) & (metrics["isi_violations_ratio"] < 0.2) & (metrics["nn_hit_rate"] > 0.80)
    print(keep_mask)

    keep_unit_ids = keep_mask[keep_mask].index.values
    keep_unit_ids = [unit_id for unit_id in keep_unit_ids]
    print(keep_unit_ids)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    0    False
    1     True
    2     True
    3     True
    4     True
    5    False
    6     True
    7     True
    8     True
    9     True
    dtype: bool
    ['1', '2', '3', '4', '6', '7', '8', '9']




.. GENERATED FROM PYTHON SOURCE LINES 61-62

And now let's create a sorting that contains only curated units and save it.

.. GENERATED FROM PYTHON SOURCE LINES 62-69

.. code-block:: Python


    curated_sorting = sorting.select_units(keep_unit_ids)
    print(curated_sorting)


    curated_sorting.save(folder="curated_sorting", overwrite=True)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    GroundTruthSorting (UnitsSelectionSorting): 8 units - 1 segments - 25.0kHz


.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div style='border:1px solid #ddd; padding:10px;'><strong>NumpyFolder (NumpyFolderSorting): 8 units - 1 segments - 25.0kHz</strong></div><details style='margin-left: 10px;'>  <summary><strong>Unit IDs</strong></summary><ul>['1' '2' '3' '4' '6' '7' '8' '9'] </details><details style='margin-left: 10px;'>  <summary><strong>Annotations</strong></summary><ul><li> <strong> name </strong>: GroundTruthSorting</li></details><details style='margin-left: 10px;'><summary><strong>Properties</strong></summary><ul><details><summary><strong>gt_unit_locations</strong></summary>[[ 3.747513   -0.10541359 14.238101  ]
     [12.6729     25.913765    9.630604  ]
     [-9.542108   15.988104   34.479168  ]
     [23.97358    -9.827189   29.84662   ]
     [23.271032    0.6536642   8.485908  ]
     [16.337154   13.090221   39.602093  ]
     [28.417702    2.1751246  46.243717  ]
     [15.713195   23.696056   34.285065  ]]</details></ul></details>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 70-71

We can also save the analyzer with only theses units

.. GENERATED FROM PYTHON SOURCE LINES 71-75

.. code-block:: Python


    clean_analyzer = analyzer.select_units(unit_ids=keep_unit_ids, format="zarr", folder="clean_analyzer")

    print(clean_analyzer)




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    SortingAnalyzer: 4 channels - 8 units - 1 segments - zarr - sparse - has recording
    Loaded 6 extensions: random_spikes, waveforms, templates, noise_levels, principal_components, quality_metrics





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.496 seconds)


.. _sphx_glr_download_tutorials_metrics_plot_4_curation.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_4_curation.ipynb <plot_4_curation.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_4_curation.py <plot_4_curation.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_4_curation.zip <plot_4_curation.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_