_____           _            __  __                                     
                                     / ____|         | |          |  \/  |                                    
                                    | |     __ _  ___| |__   ___  | \  / | ___                                
                                    | |    / _` |/ __| '_ \ / _ \ | |\/| |/ _ \                               
                                    | |___| (_| | (__| | | |  __/ | |  | |  __/                               
                                     \_____\__,_|\___|_| |_|\___| |_|  |_|\___|                               
                                                                                                              
                                                                                                              
                                  _____  __  __     __            _____                                       
                                 |_   _|/ _| \ \   / /           / ____|                                      
                                   | | | |_   \ \_/ /__  _   _  | |     __ _ _ __                             
                                   | | |  _|   \   / _ \| | | | | |    / _` | '_ \                            
                                  _| |_| |      | | (_) | |_| | | |___| (_| | | | |                           
                                 |_____|_|      |_|\___/ \__,_|  \_____\__,_|_| |_|                           
                                                                                                              
                                                                                                              
                           ___       _                      _             ___ _                               
                          / _ \ _  _| |_ _ _ _  _ _ _  _ _ (_)_ _  __ _  / __| |_____ __ __                   
                         | (_) | || |  _| '_| || | ' \| ' \| | ' \/ _` | \__ \ / _ \ V  V /                   
                          \___/ \_,_|\__|_|  \_,_|_||_|_||_|_|_||_\__, | |___/_\___/\_/\_/                    
                                                                  |___/                                       
                                __      __   _      ___                      _                                
                                \ \    / /__| |__  | _ \___ __ _ _  _ ___ __| |_ ___                          
                                 \ \/\/ / -_) '_ \ |   / -_) _` | || / -_|_-<  _(_-<                          
                                  \_/\_/\___|_.__/ |_|_\___\__, |\_,_\___/__/\__/__/                          
                                                              |_|                                             
                                                                                                       1 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                                         echo $(whoami)                                         
                                                                                                              
                                                                                                              
                                                                                                              
            •  Principle Software Engineer*                         Here is me with me cat, Fritzy            
            •  Mechanical keyboard and ergonomics enthusiast        =3                                        
            •  Die hard terminal fanboy                                                                       
            •  Functional programming nerd                                                                    
            •  Co-founder of the PHL Code Club                                                                
               ◦  I have stickers =3                                                                          
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                       2 / 20 
                                                                    *Technically I'm still a Senior           
                                                                                                              
                                                                                                              
                                          On the docket                                          
                                                                                                              
                                                                                                              
                                                                                                              
         ██ Computer memory 101                                                                               
                                                                                                              
         ▓▓▓ Average web app structure                                                                        
                                                                                                              
         ▒▒▒▒ Why cache in web services?                                                                      
                                                                                                              
         ░░░░░ Following a web request through the service                                                    
                                                                                                              
            •  Browser/client caching                                                                         
            •  CDNs and edge caching                                                                          
            •  Application caching                                                                            
                                                                                                              
         ░░░░░░ Caching strategies and pitfalls                                                               
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                       3 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                                  Cache terminology 101                                  
                                                                                                              
                                                                                                              
                                                                                                              
         ██ Common caching terms you'll hear                                                                  
                                                                                                              
         ▓▓▓ Cache outcomes                                                                                   
                                                                                                              
            •  Hit: Data found in cache - fast response                                                       
            •  Miss: Data not in cache - need to fetch from origin                                            
            •  Stale: Cached data is outdated but still served                                                
            •  Fresh: Cached data is current and valid                                                        
                                                                                                              
         ▓▓▓ Cache management                                                                                 
                                                                                                              
            •  TTL (Time To Live): How long data stays cached                                                 
            •  Invalidation: Marking cached data as outdated                                                  
            •  Eviction/Purge/Clear: Removing data from cache                                                 
            •  CDN (Content Delivery Network): Global cache network                                           
            •  Edge: Compute platforms that live in CDN Points of Presence (POP)                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                       4 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                                    Computer memory 101                                    
                                                                                                              
                                                                                                              
                                                                                                              
         ██ Types of computer memory                                                                          
                                                                                                              
         ▓▓▓ Persistent                                                                                       
                                                                                                              
            •  Long term storage                                                                              
            •  Slow (comparatively)                                                                           
            •  Does not require power to retain data                                                          
                                                                                                              
         ▓▓▓ Volatile                                                                                         
                                                                                                              
            •  Short term storage                                                                             
            •  Fast(er)                                                                                       
               ◦  Depends on the specifics of the volatile memory                                             
               ◦  RAM is faster than persistent drives thanks to bandwidth and physical location near         
                  the CPU                                                                                     
               ◦  CPU caches (HE SAID THE WORD) are faster than RAM due to colocation with the CPU,           
                  but they are tiny in comparison                                                             
            •  Requires power to retain information, once power is lost, all data is lost                     
                                                                                                              
         We'll mostly focus on persistent storage on disk/SSD and RAM for volatile memory.                    
                                                                                                              
                                                                                                       5 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                               Modern web architectures                               
                                                                                                              
                                                                                                              
                                                                                                              
         ▓▓▓ Main components for ConTracker                                                                   
                                                                                                              
         ▒▒▒▒ Client                                                                                          
                                                                                                              
            •  Web app, mobile app, or even other                                                             
               servers.                                                                                       
            •  Requests data from the API server,                                                             
               such as user profiles, messages,                                                               
               etc.                                                                                           
                                                                                                              
         ▒▒▒▒ API Server                                                                                      
                                                                                                              
            •  Small HTTP server.                          ▒▒▒▒ Storage                                       
            •  Handles requests for data from                                                                 
               clients.                                       •  Database                                     
                                                                 ◦  Stores structured data such as            
                                                                    user profiles, relationships              
                                                                    between users, etc.                       
                                                              •  Object Storage                               
                                                                 ◦  Stores large objects such as              
                                                                    photos, audio files, and videos.          
                                                                                                       6 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                        But why spend my cash on cache?                        
                                                                                                              
                                                                                                              
                                                                                                              
         ██ The problem with no caching                             ▓▓▓ The numbers that hurt                 
                                                                                                              
         Requests to the API server without caching might              •  Network latency: 50-200ms           
         look something like this:                                        per request                         
                                                                       •  Database queries:                   
                                                                          100-500ms for complex               
                                                                          operations                          
                                                                       •  Users expect responses              
                                                                          under 100ms                         
                                                                       •  Every 100ms delay = 1%              
                                                                          drop in sales [1]                   
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                       7 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                            Browser caching (automatic)                            
                                                                                                              
                                                                                                              
                                                                                                              
         Your browser is already caching lots of           ██ What gets cached automatically                  
         data automatically*:                                                                                 
                                                              •  JavaScript bundles (maybe a SPA)             
                                                              •  CSS stylesheets                              
                                                              •  Photos                                       
                                                              •  Icons                                        
                                                              •  Fonts                                        
                                                                                                              
                                                           ▓▓▓ HTTP response headers control this             
                                                           behavior                                           
                                                                                                              
                                                                                                              
                                                             # Cache for 1 day                                
                                                             Cache-Control: public, max-age=86400             
                                                             # Version identifier                             
                                                             ETag: "company-logo-v2.png"                      
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                       8 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                                Client caching (manual)                                
                                                                                                              
                                                                                                              
                                                                                                              
         We can also cache API responses in our client code.                                                  
         This is perfect for high priority data that might                                                    
         not change often, or is readily controlled by the                                                    
         user.                                                                                                
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                       9 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                       CDNs - Your global cache network                       
                                                                                                              
                                                                                                              
                                                                                                              
         ██ What do CDN's generally                                                                           
         cache                                                                                                
                                                                                                              
            •  Web app assets                                                                                 
               ◦  HTML                                                                                        
               ◦  CSS                                                                                         
               ◦  JS                                                                                          
            •  Static content                                                                                 
               ◦  Logos                                                                                       
               ◦  Fonts                                                                                       
               ◦  Public Documents                                                                            
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                      10 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
            HTTP Cache (Varnish, Nginx w/ plugins, etc)            
                                                                                                              
                                                                                                              
                                                                                                              
         HTTP reverse proxy that sits in front of ConTracker:                                                 
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
         Perfect for caching entire HTTP responses on the edge!                                               
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                      11 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
         Application caches - Redis, Memcached, Valkey,         
                                                                                                              
                                                  et al                                                  
                                                                                                              
                                                                                                              
                                                                                                              
         Where API servers store frequently accessed data:                                                    
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
         ██ How it works                                                                                      
                                                                                                              
            •  Cache key strategy: Use consistent naming like user_prefs:{user_id} and post:{post_id}         
            •  TTL management: Different timeframes for different data types                                  
            •  Cache warming: Pre-populate frequently accessed data                                           
            •  Fallback handling: Always handle cache misses gracefully                                       
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                      12 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                  Cache invalidation - The hard problem                  
                                                                                                              
                                                                                                              
                                                                                                              
         ▍ "There are only two hard things in Computer Science: cache invalidation, naming things,            
         ▍ and off-by-one errors."                                                                            
                                                                                                              
         ██ Cache invalidation strategies                                                                     
                                                                                                              
         ▓▓▓ Time-based (TTL) - Simple                                                                        
                                                                                                              
         Set expiration times when storing data. Post info gets 1 hour TTL, user sessions get 30              
         minutes. Easy to implement but might serve stale data.                                               
                                                                                                              
         ▓▓▓ Event-based - More complex but accurate                                                          
                                                                                                              
         When data changes, immediately invalidate related caches. Post edits trigger cache clears            
         for that post and all search results. Keeps data fresh but requires more coordination.               
                                                                                                              
         ▓▓▓ Manual - For emergencies                                                                         
                                                                                                              
         Admin endpoints to clear caches when things go wrong. The "nuclear option" when you need to          
         clear things quickly.                                                                                
                                                                                                              
                                                                                                              
                                                                                                      13 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                             The cache stampede problem                             
                                                                                                              
                                                                                                              
                                                                                                              
         What happens when the API's "trending posts" cache expires:                                          
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                      14 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                            The cache stampede solution                            
                                                                                                              
                                                                                                              
                                                                                                              
         ██ Lock-based approach                                                                               
                                                                                                              
         First request gets a lock, computes the data, and stores it. Other requests wait for the             
         first one to finish and return the data that is stored in the cache.                                 
                                                                                                              
         Also called cache collapse or request coalescing [2]                                                 
                                                                                                              
         ▓▓▓ Probabilistic refresh                                                                            
                                                                                                              
         Refresh cache before it expires based on random probability. Spreads out the load instead of         
         everyone hitting at once.                                                                            
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                      15 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
              Common pitfalls (and how we avoided them)              
                                                                                                              
                                                                                                              
                                                                                                              
         ██ Cache consistency nightmare                                                                       
                                                                                                              
            •  Problem: CDN shows old case status, app cache shows new status                                 
            •  Solution: Version-based invalidation and cache warming                                         
                                                                                                              
         ▓▓▓ Over-caching everything                                                                          
                                                                                                              
            •  Problem: Caching constantly updating information can lead to consistently stale data           
            •  Solution: Profile first, then cache selectively                                                
                                                                                                              
         ▒▒▒▒ Under-caching the obvious                                                                       
                                                                                                              
            •  Problem: Didn't cache document OCR operations - server melting                                 
            •  Solution: Cache expensive operations, not just database queries                                
                                                                                                              
         ░░░░░ Default TTL disaster                                                                           
                                                                                                              
            •  Problem: Used default TTL for everything                                                       
            •  Solution: Different TTLs for different data types                                              
                                                                                                              
                                                                                                              
                                                                                                      16 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                                         Best practices                                         
                                                                                                              
                                                                                                              
                                                                                                              
         ██ Do ✅                                                                                              
                                                                                                              
            •  Set appropriate TTLs: Posts = 1 hour, sessions = 30 minutes                                    
            •  Monitor cache hit ratios: Aim for >80% on critical paths                                       
            •  Use consistent cache keys: user_prefs:{user_id}, not userprefs{id}                             
            •  Handle cache failures gracefully: App should work even if cache is down                        
            •  Warm critical caches: Pre-populate high-profile users posts                                    
                                                                                                              
         ▓▓▓ Don't ❌                                                                                          
                                                                                                              
            •  Cache user-specific data globally: Don't put user preferences in CDN                           
            •  Ignore cache stampede: Popular endpoints need protection                                       
            •  Cache sensitive information: PII, sensitive documents, etc.                                    
            •  Cache everything: Caching rarely-accessed data wastes memory                                   
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                      17 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                                                Summary                                                
                                                                                                              
                                                                                                              
                                                                                                              
         ██ Start simple                                   ██ Layer strategically                             
                                                                                                              
            •  Browser cache + CDN covers 80% of              •  Each cache serves different needs            
               performance gains                                 and timeframes                               
            •  Add app cache when database becomes            •  Static assets → CDN (long TTL)               
               the bottleneck                                 •  API responses → Edge cache (medium           
            •  Add edge cache when distributing API              TTL)                                         
               responses globally                             •  Expensive computations → App cache           
            •  Don't over-engineer early                         (short TTL)                                  
                                                                                                              
         ▓▓▓ Measure everything                            ▓▓▓ Invalidate based on needs                      
                                                                                                              
            •  You can't optimize what you don't              •  TTL-based is simple but can serve            
               measure                                           stale data                                   
            •  Cache hit ratios matter more than              •  Event-based is complex but keeps             
               cache size                                        data fresh                                   
            •  Monitor P95/P99 response times, not            •  Have a manual override for                   
               just averages                                     emergencies                                  
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                      18 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                                             Questions?                                             
                                                                                                              
                                                                                                              
                                                                                                              
         ██ Citations                                                                                         
                                                                                                              
            1. Web performance affects sales (                                                                
               https://glinden.blogspot.com/2006/12/slides-from-my-talk-at-stanford.html)                     
            2. Cache Collapse/Request Coalescing (                                                            
               https://www.fastly.com/documentation/guides/concepts/edge-state/cache/request-collapsi         
               ng/)                                                                                           
                                                                                                              
         ▓▓▓ Resources for your caching journey                                                               
                                                                                                              
            •  Redis Documentation (https://redis.io/docs/) - Comprehensive guide                             
            •  Cloudflare Cache Docs (https://developers.cloudflare.com/cache/) - CDN best practices          
            •  Varnish Book (https://book.varnish-software.com/) - Advanced HTTP caching                      
            •  High Performance Browser Networking (https://hpbn.co/) - Free online book                      
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                      19 / 20 
                                                                                                              
                                                                                                              
                                                                                                              
                                     Connect with me :)                                     
                                                                                                              
                                                                                                              
                                                                                                              
         Me:                                               PHL Code Club:                                     
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                              
                                                                                                      20 / 20