Deep Learning (DL) and Partial Differential Equations (PDE)

[Daniel Duffy]

PDE are not immune to the DL revolution it seems.

Here is a random example.

Neural networks for solving differential equations – Becoming Human

My suspicions is DL for PDE is a solution looking for a problem. Both articles has major major issues on many levels.

A certain amount of intellectual rigor and honesty is needed. Caveat: part of my research was (and still is) PDE/FEM/FDM both in academia (convergence in Sobolev spaces) and industry (oil/gas etc.) in this area.

I just don't get this stuff. Much of it is very confused. (e.g. The Galerkin method went out of fashion around 1943).

 

[Pavlos Sakoglou]

PDE are not immune to the DL revolution it seems.

Here is a random example.

Neural networks for solving differential equations – Becoming Human

My thesis is DL for PDE is a solution looking for a problem. Both articles has major major issues on many levels.

A certain amount of intellectual rigor and honesty is needed. Caveat: part of my research was (and still is) PDE/FEM/FDM both in academia (convergence in Sobolev spaces) and industry (oil/gas etc.) in this area.

I just don't get this stuff. Much of it is very confused. (e.g. The Galerkin method went out of fashion around 1943).

I was in a CNN + PDE seminar today at NYU and basically this guy was describing how we can view deep learning as an ODE (stability and numerical ODE problem) as a PDE when the feature space is an image.

I couldn't grasp the full gravity of it, but here is some relevant input from the seminar:
[1705.03341] Stable Architectures for Deep Neural Networks
[1703.02009] Learning across scales - A multiscale method for Convolution Neural Networks
https://arxiv.org/pdf/1704.04932.pdf

You can also contact the guy who gave the lecture -- he is assistant professor from uni of Emory in Atlanta, he was young and very friendly. His name is Lars Ruthotto:
Lars Ruthotto | about

This might not address your question directly, but by looking into NN-PDE connections maybe you can get a better idea how to approach your problem?

Hope this is helpful.

 

[Daniel Duffy]

Nice links, Pavlos. Thanks

 

[Daniel Duffy]

Here is a ML librarys using RKHS

codpy

An RKHS based module for machine learning and data mining

pypi.org

Reproducing kernel Hilbert space - Wikipedia

en.wikipedia.org

 

[Paul Lopez]

@Daniel Duffy has deep learning come up very often in PDE solving from what you've seen? I wonder if the implementation gap in your view has improved any. Is it a decent/interesting enough path for research?

 

[Daniel Duffy]

@Daniel Duffy has deep learning come up very often in PDE solving from what you've seen? I wonder if the implementation gap in your view has improved any. Is it a decent/interesting enough path for research?

I don't think so; it is not based on reality.
No future, at least not the way CS is approaching it.

I am not a betting man, but my money would not be on it.

 

[Paul Lopez]

I don't think so; it is not based on reality.
No future, at least not the way CS is approaching it.

I am not a betting man, but my money would not be on it.

woof! Poor DL. Back to meat and potatoes basics looks like for me