Stat 654: Introduction to Applied Deep Learning
Department of Statistics and Biostatistics, CSU East Bay
Spring 2024:
Week 7:
- Final: The final for the class will be take-home. You can complete it at home individually. I will be available during the scheduled exam time for questions. See the Homework link.
- Project: Your project should be completed before the end of the semester. The last day you can submit is May 12.
- Spotlight software: Rtools
- Spotlight website: Using Pre-Trained Models
- Spotlight website: Posit Keras Using Pre-Trained Models
- Spotlight website: Tesnorflow Hub tfhub
- Code Example:
- Try out the Yolo Example from Part 3 of the Final.
- Spotlight Transformers: Hugging Face NLP Course Deep RL Course
- RStudio Keras website: Posit Tensorflow keras cheatsheet
- Spotlight blog: Posit AI Blog
- Spotlight gallery: Posit AI Gallery
- Spotlight wiki: pathmind.ai pathmind.ai glossary
- Spotlight blog: Top 10 Pretrained Models to get you Started with Deep Learning
- Spotlight blog: Using Keras’ Pre-trained Models for Feature Extraction in Image Clustering
- Spotlight blog: Common architectures in convolutional neural networks. What is ResNet?
- What are Autoencoders? ArXiv Insights Variational Autoecoders
- What are GANs? ArXiv Insights GANS
- What is Reinforcement Learning? ArXiv Insights Reinforcement Learning keras-rl
- What is Stable Diffusion? Latent Diffusion Models EXPLAINED
- What is AutoKeras? autokeras
- What are LLMs? What is a Large Language Model (LLM)?
- What is autoGPT? What is Auto-GPT? Everything to know about the next powerful AI tool
- Presentation:
- Spotlight blog: NVIDIA Upscaling
- Spotlight software: NVIDIA RTX
- Spotlight translation hardware: iflytek
- Spotlight paper: LSTM Neural Reordering Feature for Statistical Machine Translation
- Spotlight paper: Translation with Attention
- Spotlight software: magenta MusicVAE MusicML Jukebox MuseNet Riffusion
- Spotlight conference: Google I/O 2023
- Warning: “Godfather of artificial intelligence” leaves Google to talk about the tech’s potential dangers
- Warning: Titans of AI Andrew Ng and Yann LeCun oppose call for pause on powerful AI systems
- Spotlight book: Deep Learning This is a bit more advanced book.
- Spotlight talk: Deep Learning
- Spotlight Website: deeplearning.ai
- Spotlight LLM: groq.com
- Spotlight LLM: cohere Comand-R+ LLMU
- Spotlight LLM: OpenAI
Week 6:
- Homework: Homework 5 has been posted. And Homework 6 has been posted, but will not be collected.
- Project: The class Project has been assigned. Make a stylized picture.
- Spotlight blog: Visualizing Neural Networks with the Grand Tour
- YouTube: vcubingx
- Spotlight blog: Understanding LSTM Networks
- YouTube: 3Blue1Brown Neural Networks
- arXiv paper: An Introduction to Transformers
- arXiv paper: Attention Is All You Need
- Spotlight Transformers: Hugging Face
- Spotlight book: Natural Language Processing with Transformers, Revised Edition
- Spotlight book: Transformers for Natural Language Processing - Second Edition
- Spotlight software: PyTorch
- Spotlight Course: Google Machine Learning Text Classification
- Spotlight Blog: Neural Network Embeddings Explained
- Spotlight: BERT
- Spotlight paper: GloVe
- Code Examples: Chapter 11
- Case Studies: Chapter 12
- LSTM to write sentences
- Transformers
- Deep Dream Elephants
- Neural style transfer
- Generating Images
- GANs
- Google Tensorflow: Tensorflow Resources
- Spotlight blog: How to Use t-SNE Effectively
- Spotlight Conference: AI Accelerator Summit
- Spotlight Github: Neural-Style-Transfer download one picture and one style. try on CoLab.
- Example Style Transfer:
Week 5:
- Quiz: The take-home Quiz has been posted. See the Homework link.
- Midterm: The take-home Midterm will be given this week.
- Project: The Project will be given this week.
- Code Examples:
- Warning: The program introduction_convnets_pretrained.R is very very computationally intensive. Unless you have a very recent computer with lots of CPU processor cores or an nVidia graphics card and Tensorflow-gpu set up you should change the number of epochs to 5 to see how long each part of the code takes to run before trying this code.
- Presentation:
- Spotlight website: ImageNet
- Presentation:
Week 4:
- Quiz: The take-home Quiz has been posted. See the Assignments link.
- Homework: Homework 4 has been posted.
- Homework: Homework 3’s due date has been extended until next week. Due Monday April 15.
- Presentation:
- Spotlight website: kaggle dogs-vs-cats
- Code Examples:
- Spotlight YouTube: Convolutional Neural Networks Explained (CNN Visualized)
Week 3:
- Spring Break: Next week is Spring Break so there will be no class on Monday and Wednesday next week. (I will be catching up on grading.)
- Homework: Homework 3 has been posted.
- Code Example: We will look at the R code from from Chapter 4.
- Binary classification, classifying-movie-reviews.R - Colab
- Multi-class classification, classifying-newswires.R Colab
- Regression, predicting-house-prices.R - Colab
- Fashion NIST fashion01.R fashion01.R - Colab
- Presentation:
- Spotlight book: Deep Learning This is a bit more advanced book.
- Spotlight talk: Deep Learning
- Presentation:
- Evaluation.html
- Evaluation.qmd News Flash: Fitted models are for FREE and for RENT now!!!!!! Just like Amazon has done for books and Apple has done for music. Just like Microsoft and Google have tried to do for software.
- Spotlight Software: OpenAI Dall-E 2 ChatGPT
- Spotlight Software: Google Research Imagen Imagen Video Gemini
- Spotlight Software: Microsoft Bing GPT4
- Spotlight Software: Anthopic Claude 3
- Spotlight Software: Cognition Devin
- Spotlight Blog: Baidu Ernie
- Spotlight Software: rtutor.ai
- Spotlight Software: VSCode
- Spotlight Software: GitHub CoPilot
- Spotlight Website: Machine Learning Glossary
- Spotlight Blog: An overview of gradient descent optimization algorithms List of the optimizers.
- Spotlight YouTube: mlverse Gradient Descent with R
- CRAN keras: keras: R Interface to ‘Keras’
Week 2:
- Handouts: Please open or print out the Handouts from Week 2 for this week.
- Spotlight Blog: Posit AI Blog
- Homework: Homework 2 has been posted.
- RStudio Tensorflow Website: tensorflow
- Presentation:
- Presentation:
- Thank you Prof. Stone: Prof. Charles Stone My undergraduate advisor 35 years ago. Please note the second to last paragraph.
- Spotlight Paper: Leo Breiman Two Cultures
- Spotlight Wikipedia: Newton’s Method
- Spotlight Wikipedia: Newton’s Method for optimization
- Spotlight Wikipedia: Stochastic gradient descent
- Spotlight Blog: sentdex Difference between a Batch and an Epoch
- RStudio Tensorflow Website: tensorflow
- Google Colab to run R: https://colab.research.google.com/notebook#create=true&language=r
- Code Example: We will look at the R code from from Chapter 4.
- Binary classification, classifying-movie-reviews.R - CoLab
- Multi-class classification, classifying-newswires.R CoLab
- Regression, predicting-house-prices.R - CoLab
Week 1:
- Class online: Wedenesday March 13 class online.
- Book: From the University Library > Databases A-Z > O > O’Reilly Then login using your university login. Find Deep Learning with R, Second Edition. Or the Python version - Book, audio book, or video book.
- Book: mdsr2e
- Book: r4ds2e
- Homework: Homework 1 has been posted.
- Presentation:
- Introductory Examples from my JSM 2018 Poster.
- R Project: Poster.zip
- cars Example
- concrete Example images.pdf
- Software Spotlight: posit Cloud Instead of dealing with Java (on Windows you need the Windows Offline (64 bit) version) to run rWeka and h2o, you could use the posit Cloud.
- iris Example
- iris01.nb.html
- iris01.nb.docx
- iris01.Rmd Corrected: perf$acc should just be perf.
- iris01.R - Colab
- Poster:
- Spotlight Blog:
- Spotlight Book: Deep Learning
- Spotlight Website: deeplearning.ai
- Spotlight Website: fast.ai
- Spotlight Website: Machine Learning Crash Course
- RStudio Tensorflow Website: tensorflow
- Google Colab to run R: https://colab.research.google.com/notebook#create=true&language=r
- YouTube:
Week 0.5:
- YouTube:
- Links:
- Links: OpenAI
- Links: Microsoft
- bingChat aka CoPilot
- Microsoft CoPilot Designer
- Links: GitHub
- Links: Google
- Links:
- Links:
- Links:
Week 0:
Learning R:
Learning Python:
Learn SQL:
Excellent References:
Data Science:
- Socviz
- r4ds2e
- ModernDive
- Yarrr!
- R Data Science Essentials
- Python Data Science Essentials
- Deep Learning Made Easy with R
- Doing Data Science
- Data Science from Scratch
- What is Data Science? (fast easy read)
- Ethics and Data Science (fast easy read)
- Data Driven (fast easy read)
- R Markdown: The Definitive Guide
Reading related to the Digital Economy:
- The Second Machine Age: Work, Progress and Prosperity in a Time of Brilliant Technologies
- Race Against the Machine
- Wired For Innovation
- Strategies for e-business success
- Understanding the Digital Economy
More Big Picture:
- Fourth Paradigm of Science: Data-Intensive Scientific Discovery
- McKinsey Global Institute Big Data: The next frontier for innovation, competition, and productivity
Tech papers:
- Experimental evidence of massive-scale emotional contagion through social networks
- High-Resolution Image Synthesis with Latent Diffusion Models
- Attention Is All You Need
- A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
- Music can be reconstructed from human auditory cortex activity using nonlinear decoding models