Exercise 1 - Tensors

In Chapter 1 of the lecture, we recapitulated tensor notation and tensor analysis. In this exercise, we will gain some more confidence in working with tensors by hand calculations as well as code.

After studying this chapter and finishing the exercise, you should be able to

• ... distinguish three types of structural optimization problems.
• ... perform tensor algebra computations such as inner products, cross products, outer products, dyadic products, and tensor contractions.
• ... apply differential operators such as divergence and gradient on tensor fields.

Task 1.0: Installation

To solve the tasks with code, we will use a package called torch-fem. It is a differentiable finite element solver based on the PyTorch framework. PyTorch is a powerful Python package to operate on tensors. In comparison to NumPy, it stores gradients together with tensors and thus allows automatic differentiation. The package is used widely for machine learning and optimization.

For installation it is best to create a new conda environment via

conda create -n "struct_opt" python

and activate that environment via

conda activate struct_opt

to have a fresh new independent virtual Python environment to install the required packages with this course. It is highly recommended to use such an environment to prevent potential conflicts with other Python projects.

In the activated environment, you should install the package torch-fem

pip install torch-fem

to install the required packages. After that, you should be able to import the torch package in this Jupyter Notebook:

import torch

torch.set_default_dtype(torch.double)

Task 1.1: Vector products

Given two vectors $\mathbf{a}, \mathbf{b} \in \mathcal{R}^3$ with $$ \mathbf{a} = \begin{pmatrix}2\\1\\3\end{pmatrix} \quad \mathbf{b} = \begin{pmatrix}5\\0\\1\end{pmatrix} $$ define the vectors in torch and compute the dot product, cross product and outer product.

Task 1.2 - Tensor products

Given the tensors $\mathbf{A}, \mathbf{B} \in \mathcal{R}^{3 \times 3}$ and $\mathbb{C} \in \mathcal{R}^{3 \times 3 \times 3 \times 3}$ convert the following expressions to sums of components and determine the dimensions of the resulting tensor.

Example:

$$\mathbf{A} \cdot \mathbf{b} \rightarrow \sum_{i,j} A_{ij}b_j \mathbf{e}_j$$

a) $$\mathbf{a} \cdot \mathbf{A} \cdot \mathbf{b}$$ b) $$\mathbf{b} \cdot \mathbf{A} \cdot \mathbf{a}$$ c) $$\mathbf{A} \cdot \mathbf{B} \cdot \mathbf{b}$$ d) $$(\mathbf{A} : \mathbf{B}) \mathbf{b}$$ e) $$(\mathbf{a} \otimes \mathbf{b}) : \mathbf{B}$$ f) $$\mathbf{A} \otimes \mathbb{C} : \mathbf{B}$$

Convert the following expressions to symbolic notation and determine the dimensions of the resulting tensor:

g) $$\sum_{z,j} A_{zj}b_z \mathbf{e}_j$$ h) $$\sum_{i,j,k} A_{ij}B_{jk}a_k \mathbf{e}_i$$ i) $$\sum_{m,n,o,p,i} C_{mnop}A_{po}\delta_{ni}a_{i} \mathbf{e}_m$$

Given the values $$ \mathbf{A} = \begin{pmatrix} 6 & 2 & 1\\ 4 & 7 & 6\\ 0 & 2 & 9 \end{pmatrix} \quad \mathbf{B} = \begin{pmatrix} 5 & 7 & 11\\ 0 & 4 & 3\\ 1 & 2 & 9 \end{pmatrix} \quad C_{ijkl} = 1 \forall i,j,k,l $$ define the tensors in torch and compute the expressions above. Reuse $\mathbf{a}$ and $\mathbf{b}$ from the first task.

Tips:

• What we denote with $\cdot$ in the lecture, can be written with an @ or torch.tensordot(...,dim=1) in numpy and torch.
• What we denote with $:$ in the lhe lecture, can be written with torch.tensordot in numpy and torch.
• Multiplication between scalars is done simply by *.
• We can use torch.einsum() to define arbitrary expressions using Einstein's summation convention. Here, the function automatically sums over indices in an expression, e.g. torch.einsum("ij,j->i",A,b) computes $\sum_{ij} A_{ij}b_j \mathbf{e}_i$

Task 1.3: Gradients in 1D

Given the function $g: \mathbf{R} \rightarrow \mathbf{R}$ defined as

$$ g(x) = x^2+x+1 $$

define the function, compute its gradient and plot it on $x \in [-5, 5]$.

Task 1.4: Gradients in 2D

Given the vectorfield $f: \mathcal{R}^2 \rightarrow \mathcal{R}$ defined as

$$ f(\mathbf{x}) = (\mathbf{x} - \tilde{\mathbf{x}}) \cdot \mathbf{Q} \cdot (\mathbf{x} - \tilde{\mathbf{x}}) $$ with $$ \mathbf{Q} = \begin{pmatrix} 2 & 1 \\ 1 & 1 \end{pmatrix} \quad \text{and} \quad \tilde{\mathbf{x}} = \begin{pmatrix} -1\\ 1 \end{pmatrix} $$ compute the gradient analytically.

Doing these computations by hand takes a while. Therefore we take a look at how to compute gradients using PyTorch. To do so, we start by defining $\mathbf{Q}$, $\tilde{\mathbf{x}}$ and the function $f(\mathbf{x})$. The function $f(\mathbf{x})$ can be implemented in a straight forward way and you should try a straight forward implementation first.

However, we would like to be able to evaluate the function for many values of $\mathbf{x}$ at the same time. This is equivalent to passing a tensor of the shape $\mathcal{R}^{... \times 2}$ with arbitray dimensions except the last axis. This can be implemented using an ellipsis ... in torch.einsum().

If your function is defined correctly, the following cell should plot the function values as a contour plot. Make sure that you have the file utils.py located in the same directory as this notebook to use the plot_contours() function.

from utils import plot_contours

# Define x
x0 = torch.linspace(-3, 3, steps=100, requires_grad=True)
x1 = torch.linspace(-3, 3, steps=100, requires_grad=True)
x = torch.stack(torch.meshgrid(x0, x1, indexing="xy"), dim=2)

plot_contours(x, f(x), title="f(x)")

Note that the requires_grad=True argument defines that these specific tensors will be used in gradient computations. They reserve storage for the tensor data as well as the gradients. Now, lets compute the actual gradients with automatic differentiation:

dfdx = torch.autograd.grad(f(x).sum(), x)[0]


# Reproduce basic plot
plot_contours(x, f(x), title="f(x)")

# Plot gradient vectors as arrows on top of previous plot
with torch.no_grad():
    stride = 5
    plt.quiver(
        x[::stride, ::stride, 0],
        x[::stride, ::stride, 1],
        dfdx[::stride, ::stride, 0],
        dfdx[::stride, ::stride, 1],
    )
    plt.axis("equal")