Python Async Patterns: Deep Dive into Coroutines and Event Loops

Sat, 25 Jul 2026 00:00:00 +0000

1. Async Programming Basics

Python’s asyncio library provides an event-loop-based coroutine concurrency model, ideal for I/O-bound and high-concurrency applications. Understanding coroutines, the event loop, and task scheduling is key to mastering asynchronous programming.

2. Core Example

The following example demonstrates writing concurrent API requests using async/await, and executing multiple coroutines in parallel with asyncio.gather.

Python Async Example

 
 
 
 

 python
 
 
import asyncio
import aiohttp
from typing import List

async def fetch_url(session: aiohttp.ClientSession, url: str) -> dict:
 """Fetch URL content asynchronously"""
 async with session.get(url) as response:
 return await response.json()

async def fetch_all(urls: List[str]) -> List[dict]:
 """Fetch all URLs concurrently"""
 async with aiohttp.ClientSession() as session:
 tasks = [fetch_url(session, url) for url in urls]
 return await asyncio.gather(*tasks, return_exceptions=True)

async def main():
 urls = [
 "https://api.example.com/users",
 "https://api.example.com/posts",
 "https://api.example.com/comments",
 ]
 results = await fetch_all(urls)
 for i, result in enumerate(results):
 status = "Success" if not isinstance(result, Exception) else "Failed"
 print(f"Request {i}: {status}")

if __name__ == "__main__":
 asyncio.run(main())

Run Script

 
 bash
 
#!/bin/bash
# Async program runner
python3 -m pip install aiohttp # Install dependencies
python3 async_fetch.py # Run async program

3. Best Practices

Use asyncio.create_task for background tasks, asyncio.Semaphore to control concurrency, and asyncio.timeout for timeout management. Avoid CPU-bound operations inside coroutines; use concurrent.futures for parallel computation instead.

Distributed Database Sharding Strategies and Consistency Guarantees

Sat, 25 Jul 2026 00:00:00 +0000

1. Sharding Strategies

When a single database reaches the terabyte scale, sharding becomes inevitable. Common partitioning strategies include hash sharding, range sharding, and consistent hashing. Choosing the right shard key is the core of the design.

2. Go Sharding Implementation

The following example demonstrates a consistent hash sharding router implemented in Go, responsible for routing requests to the correct database shard.

Go Sharding Logic

 
 
 
 

 go
 
 
package shard

import (
	"crypto/sha256"
	"fmt"
	"hash/crc32"
)

type Router struct {
	shards []string
}

func NewRouter(nodes []string) *Router {
	return &Router{shards: nodes}
}

// ShardKey calculates the target shard based on user ID
func (r *Router) ShardKey(userID string) string {
	h := crc32.ChecksumIEEE([]byte(userID))
	idx := int(h) % len(r.shards)
	return r.shards[idx]
}

// ShardKeyConsistent uses consistent hashing
func (r *Router) ShardKeyConsistent(userID string) string {
	hash := sha256.Sum256([]byte(userID))
	value := int(hash[0])<<24 | int(hash[1])<<16 |
		int(hash[2])<<8 | int(hash[3])
	idx := value % len(r.shards)
	// Virtual node support (simplified)
	return r.shards[idx]
}

func main() {
	nodes := []string{"shard-0", "shard-1", "shard-2", "shard-3"}
	router := NewRouter(nodes)

	userIDs := []string{"user_1001", "user_1002", "user_1003"}
	for _, uid := range userIDs {
		shard := router.ShardKeyConsistent(uid)
		fmt.Printf("User %s -> %s\n", uid, shard)
	}
}

SQL Query Example

 
 
 
 

 sql
 
 
-- Create shard table
CREATE TABLE IF NOT EXISTS orders_0000 (
 id BIGINT PRIMARY KEY,
 user_id VARCHAR(64) NOT NULL,
 amount DECIMAL(12,2) NOT NULL,
 created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
 INDEX idx_user_id (user_id)
);

-- Cross-shard query (results merged at application layer)
-- The query runs independently on each shard
SELECT user_id, SUM(amount) as total
FROM orders_0000
WHERE created_at >= '2026-01-01'
GROUP BY user_id;

3. Consistency Guarantees

For distributed transactions, TCC or Saga patterns are recommended, combined with eventual consistency compensation mechanisms. For read-heavy scenarios, use read-write separation and cache warming strategies to improve performance.

Software Development on SRE & AI Field Notes