Palmer Penguins kNN Classification (Python)

This Quarto document runs the Python translation of kNN_example_penguins.qmd through reticulate.

library(reticulate)
py_require(c("pandas", "numpy", "scikit-learn", "matplotlib",
             "seaborn", "palmerpenguins"))

Load and inspect the data

from pathlib import Path

import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
from sklearn.metrics import accuracy_score, ConfusionMatrixDisplay, confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier

DATA_FILE = Path("penguins.csv")

if DATA_FILE.exists():
    penguins = pd.read_csv(DATA_FILE)
else:
    try:
        from palmerpenguins import load_penguins
        penguins = load_penguins()
    except ImportError:
        penguins = sns.load_dataset("penguins")

penguins = penguins.drop(columns=["island", "sex"])
print(penguins.head())

  species  bill_length_mm  bill_depth_mm  flipper_length_mm  body_mass_g
0  Adelie            39.1           18.7              181.0       3750.0
1  Adelie            39.5           17.4              186.0       3800.0
2  Adelie            40.3           18.0              195.0       3250.0
3  Adelie             NaN            NaN                NaN          NaN
4  Adelie            36.7           19.3              193.0       3450.0

print(penguins["species"].value_counts().sort_index())

species
Adelie       152
Chinstrap     68
Gentoo       124
Name: count, dtype: int64

print(penguins.isna().sum())

species              0
bill_length_mm       2
bill_depth_mm        2
flipper_length_mm    2
body_mass_g          2
dtype: int64

Visualize the data

sns.scatterplot(data=penguins, x="bill_length_mm", y="bill_depth_mm")
plt.tight_layout()
plt.show()

sns.scatterplot(
    data=penguins, x="bill_length_mm", y="bill_depth_mm", hue="species"
)
plt.tight_layout()
plt.show()

g = sns.FacetGrid(penguins, col="species")
g.map_dataframe(sns.scatterplot, x="bill_length_mm", y="bill_depth_mm")

g.tight_layout()

plt.show()

Fit and evaluate the kNN model

The original tutorial uses an unweighted kNN model with k = 4. Python’s scikit-learn requires complete predictor rows, so missing rows are removed before fitting.

penguins_complete = penguins.dropna().copy()

X = penguins_complete.drop(columns="species")
y = penguins_complete["species"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.20, stratify=y, random_state=123
)

knn_model = KNeighborsClassifier(n_neighbors=4, weights="uniform")
knn_model.fit(X_train, y_train)

KNeighborsClassifier(n_neighbors=4)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

KNeighborsClassifier

?Documentation for KNeighborsClassifieriFitted

Parameters

	n_neighbors n_neighbors: int, default=5 Number of neighbors to use by default for :meth:`kneighbors` queries.	4
	weights weights: {'uniform', 'distance'}, callable or None, default='uniform' Weight function used in prediction. Possible values: - 'uniform' : uniform weights. All points in each neighborhood are weighted equally. - 'distance' : weight points by the inverse of their distance. in this case, closer neighbors of a query point will have a greater influence than neighbors which are further away. - [callable] : a user-defined function which accepts an array of distances, and returns an array of the same shape containing the weights. Refer to the example entitled :ref:`sphx_glr_auto_examples_neighbors_plot_classification.py` showing the impact of the `weights` parameter on the decision boundary.	'uniform'
	algorithm algorithm: {'auto', 'ball_tree', 'kd_tree', 'brute'}, default='auto' Algorithm used to compute the nearest neighbors: - 'ball_tree' will use :class:`BallTree` - 'kd_tree' will use :class:`KDTree` - 'brute' will use a brute-force search. - 'auto' will attempt to decide the most appropriate algorithm based on the values passed to :meth:`fit` method. Note: fitting on sparse input will override the setting of this parameter, using brute force.	'auto'
	leaf_size leaf_size: int, default=30 Leaf size passed to BallTree or KDTree. This can affect the speed of the construction and query, as well as the memory required to store the tree. The optimal value depends on the nature of the problem.	30
	p p: float, default=2 Power parameter for the Minkowski metric. When p = 1, this is equivalent to using manhattan_distance (l1), and euclidean_distance (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used. This parameter is expected to be positive.	2
	metric metric: str or callable, default='minkowski' Metric to use for distance computation. Default is "minkowski", which results in the standard Euclidean distance when p = 2. See the documentation of `scipy.spatial.distance <https://docs.scipy.org/doc/scipy/reference/spatial.distance.html>`_ and the metrics listed in :class:`~sklearn.metrics.pairwise.distance_metrics` for valid metric values. If metric is "precomputed", X is assumed to be a distance matrix and must be square during fit. X may be a :term:`sparse graph`, in which case only "nonzero" elements may be considered neighbors. If metric is a callable function, it takes two arrays representing 1D vectors as inputs and must return one value indicating the distance between those vectors. This works for Scipy's metrics, but is less efficient than passing the metric name as a string.	'minkowski'
	metric_params metric_params: dict, default=None Additional keyword arguments for the metric function.	None
	n_jobs n_jobs: int, default=None The number of parallel jobs to run for neighbors search. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary <n_jobs>` for more details. Doesn't affect :meth:`fit` method.	None

Fitted attributes

Name	Type	Value
classes_ classes_: array of shape (n_classes,) Class labels known to the classifier	ndarray[object](3,)	['Adelie','Chinstrap','Gentoo']
effective_metric_ effective_metric_: str or callble The distance metric used. It will be same as the `metric` parameter or a synonym of it, e.g. 'euclidean' if the `metric` parameter set to 'minkowski' and `p` parameter set to 2.	str	'eu...an'
effective_metric_params_ effective_metric_params_: dict Additional keyword arguments for the metric function. For most metrics will be same with `metric_params` parameter, but may also contain the `p` parameter value if the `effective_metric_` attribute is set to 'minkowski'.	dict	{}
feature_names_in_ feature_names_in_: ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0	ndarray[object](4,)	['bill_length_mm','bill_depth_mm','flipper_length_mm','body_mass_g']
n_features_in_ n_features_in_: int Number of features seen during :term:`fit`. .. versionadded:: 0.24	int	4
n_samples_fit_ n_samples_fit_: int Number of samples in the fitted data.	int	273
outputs_2d_ outputs_2d_: bool False when `y`'s shape is (n_samples, ) or (n_samples, 1) during fit otherwise True.	bool	False

def evaluate(X_data, y_data, label):
    predictions = knn_model.predict(X_data)
    cm = confusion_matrix(y_data, predictions, labels=knn_model.classes_)
    print(f"{label} accuracy: {accuracy_score(y_data, predictions):.4f}")
    print(pd.DataFrame(cm, index=knn_model.classes_, columns=knn_model.classes_))
    ConfusionMatrixDisplay(cm, display_labels=knn_model.classes_).plot(cmap="Blues")
    plt.title(f"{label} confusion matrix")
    plt.tight_layout()
    plt.show()
    return predictions, cm

train_predictions, train_cm = evaluate(X_train, y_train, "Training")

Training accuracy: 0.8681
           Adelie  Chinstrap  Gentoo
Adelie        118          1       2
Chinstrap      27         25       2
Gentoo          4          0      94

test_predictions, test_cm = evaluate(X_test, y_test, "Testing")

Testing accuracy: 0.8841
           Adelie  Chinstrap  Gentoo
Adelie         30          0       0
Chinstrap       6          8       0
Gentoo          1          1      23