A central question in machine learning theory is whether an algorithm enjoys good generalization, which is the ability to correctly predict new examples from prior observations. While classical wisdom advocates for methods with fewer parameters than data points in order to avoid overfitting, modern machine learning algorithms are severely over-parameterized and perfectly fit training data. In this talk, we study the generalization error of a simple and ubiquitous over-parameterized algorithm: we select a function belonging to a function space of dimension p that interpolates n prescribed data points by minimizing a chosen weighted norm, where p >> n. Under natural and general conditions, we prove that both the interpolants and their generalization errors converge as the number of parameters grow, and the limiting interpolant belongs to a reproducing kernel Hilbert space. This rigorously establishes an implicit bias of minimum weighted norm interpolation and explains why norm minimization may benefit from over-parameterization. As special cases of this theory, we study interpolation by trigonometric polynomials and spherical harmonics. Our approach is from a deterministic and approximation theory viewpoint, as opposed to a statistical or random matrix one.