In this article, we visit PINNsFormer, a Transformer-based physics-informed framework that blends attention mechanisms with wavelet activations to model complex spatiotemporal dynamics.
The wavelet activation function is genuinely clever, way more than just another custom activation. The learnable w1 and w2 params letting the network adjust amplitide and phase during training is such a clean approach to capturing oscillatory PDE behaviors. We ran into similar gradient instability issues on some fluid dynamics problems, and this makes me wanna test wavelet activations outside the PINNs context entirely.
Yes, Wavelet Function can be great for problems outside the PINNs domain as well. Let us know if you get some interesting results, I would love to learn that as well
The wavelet activation function is genuinely clever, way more than just another custom activation. The learnable w1 and w2 params letting the network adjust amplitide and phase during training is such a clean approach to capturing oscillatory PDE behaviors. We ran into similar gradient instability issues on some fluid dynamics problems, and this makes me wanna test wavelet activations outside the PINNs context entirely.
Hello @Neural Foundry,
Yes, Wavelet Function can be great for problems outside the PINNs domain as well. Let us know if you get some interesting results, I would love to learn that as well