{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Numpy Gems, Part 3\n",
"\n",
"Much of scientific computing revolves around the manipulation of indices. Most formulas involve sums of things and at the core of it the formulas differ by which things we're summing.\n",
"\n",
"Being particularly clever about indexing helps with that. A complicated example is the [FFT](https://en.wikipedia.org/wiki/Cooley%E2%80%93Tukey_FFT_algorithm). A less complicated example is computing the inverse of a permutation:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"np.random.seed(1234)\n",
"x = np.random.choice(10, replace=False, size=10)\n",
"s = np.argsort(x)\n",
"inverse = np.empty_like(s)\n",
"inverse[s] = np.arange(len(s), dtype=int)\n",
"np.all(x == inverse)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The focus of this post is to expand on an extremely useful, vectorizable isomorphism between indices, that comes up all the time: indexing pairs. In particular, it's often the case that we'd want to come up with an _a priori_ indexing scheme into a weighted, complete undirected graph on $V$ vertices and $E$ edges.\n",
"\n",
"In particular, our edge set is $\\binom{[V]}{2}=\\left\\{(0, 0), (0, 1), (0, 2), \\cdots, (V-3, V-2), (V-3, V-1), (V-2, V-1)\\right\\}$, the set of ordered $2$-tuples. Our index set is $\\left[\\binom{V}{2}\\right]=\\{0, 1, 2, \\cdots, V(V-1)/2 - 1\\}$ (note we're 0-indexing here).\n",
"\n",
"Can we come up with an isomorphism between these two sets that vectorizes well?\n",
"\n",
"A natural question is why not just use a larger index. Say we're training a [GGNN](https://arxiv.org/abs/1511.05493), and we want to maintain embeddings for our edges. Our examples might be in a format where we have two vertices $(v_1, v_2)$ available. We'd like to index into an edge array maintaining the corresponding embedding. Here, you may very well get away with using an array of size $V^2$. That takes about twice as much memory as you need, though.\n",
"\n",
"A deeper problem is simply that you can _represent_ invalid indices, and if your program manipulates the indices themselves, this can cause bugs. This matters in settings like [GraphBLAS](http://graphblas.org/) where you're trying to vectorize classical graph algorithms.\n",
"\n",
"The following presents a completely static isomorphism that doesn't need to know `V` in advance."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# an edge index is determined by the isomorphism from\n",
"# ([n] choose 2) to [n choose 2]\n",
"\n",
"# drop (i, j) to (i, j - i - 1) first. then:\n",
"\n",
"# (0, 0) (0, 1) (0, 2)\n",
"# (1, 0) (1, 1)\n",
"# (2, 0)\n",
"\n",
"# isomorphism goes in downward diagonals\n",
"# like valence electrons in chemistry\n",
"\n",
"def c2(n):\n",
" return n * (n - 1) // 2\n",
"\n",
"def fromtup(i, j):\n",
" j = j - i - 1\n",
" diagonal = i + j\n",
" return c2(diagonal + 1) + i\n",
"\n",
"def totup(x):\n",
" # https://math.stackexchange.com/a/1417583 + some int/float rewriting\n",
" diagonal = (1 + np.sqrt(8 * x + 1).astype(np.uint64)) // 2 - 1\n",
" i = x - c2(diagonal + 1)\n",
" j = diagonal - i\n",
" j = j + i + 1\n",
" return i, j\n",
"\n",
"nverts = 1343\n",
"edges = np.arange(c2(nverts), dtype=int)\n",
"np.all(fromtup(*totup(edges)) == edges)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This brings us to our first numpy gem of this post, to check that our isomorphism is surjective, `np.triu_indices`."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"left, right = totup(edges)\n",
"expected_left, expected_right = np.triu_indices(nverts, k=1)\n",
"from collections import Counter\n",
"Counter(zip(left, right)) == Counter(zip(expected_left, expected_right))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The advantage over indexing into `np.triu_indices` is of course the scenario where you _don't_ want to fully materialize all edges in memory, such as in frontier expansions for graph search.\n",
"\n",
"You might be wondering how dangerous that `np.sqrt` is, especially for large numbers. Since we're concerned about the values of `np.sqrt` for inputs at least 1, and on this domain the mathematical function is sublinear, there's actually _less_ rounding error in representing the square root of an integer with a double than the input itself. [Details here](https://stackoverflow.com/a/22547057/1779853).\n",
"\n",
"Of course, we're in trouble if `8 * x + 1` cannot even up to ULP error be represented by a 64-bit double. It's imaginable to have graphs on `2**32` vertices, so it's not a completely artificial concern, and in principle we'd want to have support for edges up to index value less than $\\binom{2^{32}}{2}=2^{63} - 2^{32}$. Numpy correctly refuses to perform the mapping in this case, throwing on `totup(2**61)`.\n",
"\n",
"In this case, some simple algebra and recalling that we don't need a lot of precision anyway will save the day."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = 2**53\n",
"float(8 * x + 1) == float(8 * x)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def totup_flexible(x):\n",
" x = np.asarray(x)\n",
" assert np.all(x <= 2 ** 63 - 2**32)\n",
" if x > 2 ** 53:\n",
" s = np.sqrt(2) * np.sqrt(x)\n",
" s = s.astype(np.uint64)\n",
" # in principle, the extra multiplication here could require correction\n",
" # by at most 1 ulp; luckily (s+1)**2 is representable in u64\n",
" # because (sqrt(2)*sqrt(2**63 - 2**32)*(1+3*eps) + 1) is (just square it to see)\n",
" s3 = np.stack([s - 1, s, s + 1]).reshape(-1, 3)\n",
" s = 2 * s3[np.arange(len(s3)), np.argmin(s3 ** 2 - 2 * x, axis=-1)]\n",
" else:\n",
" s = np.sqrt(8 * x + 1).astype(np.uint64)\n",
" add = 0 if x > 2 ** 53 else 1\n",
" diagonal = (1 + s) // 2 - 1\n",
" diagonal = diagonal.reshape(x.shape)\n",
" i = x - c2(diagonal + 1)\n",
" j = diagonal - i\n",
" j = j + i + 1\n",
" return i, j\n",
"\n",
"x = 2 ** 63 - 2 ** 32\n",
"fromtup(*totup_flexible(x)) == x"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}