TechWhale

Ultimate Full-Stack Development Workflow: BMAD + Claude Code + Spec Kit + MCP + Shadcn

Mayur Chavhan — Tue, 09 Sep 2025 19:02:00 GMT

Overview: The Perfect AI Development Stack

This comprehensive guide combines five powerful frameworks to create the ultimate AI-driven development experience:

BMAD Method: Specialized AI agents working as a development team
Spec Kit: Specification-driven development methodology
Claude Code: AI-powered IDE with agent orchestration
MCP Memory: Persistent context management across sessions
Shadcn UI: Modern component library with AI integration

🚀 Complete Setup Process

Step 1: Core Framework Installation

Install BMAD Method

# In your project directory
npx bmad-method install

Install Spec Kit

# Initialize with Claude Code support
uvx --from git+https://github.com/github/spec-kit.git specify init <PROJECT_NAME> --ai claude

Configure Claude Code with Kimi K2 API

Set up Claude Code with your Kimi K2 API configuration in the settings.

Step 2: MCP Memory Integration

Install Memory Keeper MCP

# Clone and setup memory persistence
git clone https://github.com/mkreyman/mcp-memory-keeper.git
cd mcp-memory-keeper
npm install && npm run build

# Add to Claude Code
claude mcp add memory-keeper npx mcp-memory-keeper

Install OpenMemory MCP (Alternative)

# For shared memory across tools
git clone https://github.com/doobidoo/mcp-memory-service.git
cd mcp-memory-service
python install.py
uv run memory server

Step 3: Shadcn UI MCP Setup

Install Shadcn MCP Server

# Add to Claude Code for automated component installation
claude mcp add shadcn-ui npx @shadcn/ui-mcp-server

Initialize Shadcn in Your Project

npx shadcn@latest init

🧠 Integrated Workflow Process

Phase 1: Specification-Driven Planning

Start with Spec Kit Requirements

# In Claude Code, use Spec Kit commands
/specify Build a full-stack e-commerce platform with product management, 
user authentication, shopping cart, and payment integration. Focus on 
modern UX with beautiful components and responsive design.

Create Technical Plan

/plan Use React with TypeScript frontend, Node.js/Express backend, 
PostgreSQL database, JWT authentication, Stripe payments, and 
shadcn/ui components for beautiful UI design.

Break Down into Tasks

/tasks Create actionable development tasks with proper sequencing

Phase 2: BMAD Agent Orchestration

Analyst Phase - Deep Requirements Analysis

# Switch to BMAD workflow
*analyst
Analyze the Spec Kit requirements and create comprehensive project brief 
for e-commerce platform with focus on beautiful UI and solid architecture.

Project Manager Phase - Strategic Planning

*pm
Create detailed PRD from analyst brief, incorporating Spec Kit specifications
and ensuring feature prioritization aligns with technical constraints.

Architect Phase - System Design

*architect
Design scalable full-stack architecture using React/TypeScript, Node.js,
PostgreSQL, and shadcn/ui components. Include API design, database schema,
and deployment strategy.

Phase 3: Context-Engineered Development

Document Sharding for Focused Development

*po
Shard PRD and Architecture documents into focused, actionable pieces
for efficient agent consumption and context management.

Memory-Enhanced Story Creation

# Use Memory MCP to maintain context
*sm
Create detailed story for user authentication system. Remember our 
architecture decisions about JWT tokens and PostgreSQL schema design.

# Memory automatically preserves:
# - Previous architectural decisions
# - Database schema choices
# - Component library selections
# - API design patterns

Shadcn-Enhanced Development

*dev
Implement authentication UI using shadcn components. Use the MCP server
to automatically install and configure needed components like forms,
buttons, and input fields.

# Shadcn MCP automatically:
# - Installs required components
# - Provides TypeScript props
# - Ensures design consistency
# - Handles component dependencies

🎯 Advanced Integration Features

Multi-Agent Collaboration with Memory

Cross-Session Context Preservation

# Before hitting context limits
context_prepare_compaction
# Get checkpoint key: "auth_system_v1_2024"

# In new session
context_restore_from_checkpoint auth_system_v1_2024
# Full context restored including architectural decisions

Git-Aware Memory Channels

# Automatically organize context by git branch
git checkout feature/payment-system
# Memory MCP creates separate context channel
# No mixing of feature contexts

Automated Component Integration

Shadcn Component Discovery

# Claude Code with Shadcn MCP can now:
- Query component registry directly
- Install components with proper configurations
- Access up-to-date component documentation
- Handle component dependencies automatically

Beautiful UI Development

*dev
Create a product catalog page using shadcn components. Include:
- Card components for product display
- Skeleton loaders for better UX
- Responsive grid layouts
- Modern typography and spacing

# MCP server handles:
# - Component installation
# - TypeScript integration  
# - Design system consistency
# - Accessibility features

Self-Correcting Code Architecture

QA Agent with Memory Context

*qa
Review the authentication implementation against our architectural standards.
Check for security best practices, TypeScript compliance, and shadcn
component usage patterns.

# QA agent remembers:
# - Previous code review feedback
# - Established coding standards
# - Security requirements
# - Component usage patterns

Iterative Refinement with Context

# Memory preserves lessons learned
context_add_insight "Authentication forms should always use zod validation 
with shadcn form components for consistent error handling"

# Future development automatically applies insights
*dev
Create user registration form using established patterns
# Automatically applies zod validation + shadcn forms

🔧 Project Structure for Integrated Workflow

your-project/
├── .bmad/                    # BMAD Method configuration
│   ├── agents/              # Specialized agent configs
│   └── workflows/           # Development workflows
├── specs/                   # Spec Kit specifications
│   ├── 001-auth-system/     # Feature specifications
│   └── 002-product-catalog/ # More features
├── memory/                  # MCP Memory storage
│   ├── context.db          # SQLite memory database
│   └── checkpoints/        # Context checkpoints
├── docs/                   # BMAD planning documents
│   ├── prd/                # Product requirements (sharded)
│   ├── architecture/       # Architecture docs (sharded)
│   └── briefs/             # Analysis briefs
├── stories/                # BMAD development stories
│   ├── epic-1-auth/        # Authentication epic
│   ├── epic-2-products/    # Product management epic
│   └── epic-3-payments/    # Payment processing epic
├── src/                    # Application source code
│   ├── components/         # Shadcn UI components
│   │   ├── ui/            # Base shadcn components
│   │   └── custom/        # Custom components
│   ├── pages/             # Application pages
│   ├── api/               # Backend API routes
│   └── lib/               # Utilities and configurations
└── package.json           # Dependencies including shadcn

🌟 Key Integration Benefits

Context Preservation Across All Tools

Memory MCP: Maintains decisions, patterns, and progress across sessions
BMAD Stories: Self-contained context files with complete implementation details
Spec Kit: Persistent specifications that evolve with project needs

Beautiful UI Development

Shadcn MCP: Automated component installation and configuration
Design Consistency: AI agents understand and apply design system patterns
Modern Components: Access to latest UI components with proper TypeScript support

Self-Correcting Architecture

QA Agent: Reviews code against established patterns and standards
Memory Insights: Preserves lessons learned for future development
Iterative Improvement: Each cycle improves code quality and consistency

Scalable Development Process

Feature Division: Large features broken into focused stories
Agent Specialization: Each agent focuses on their expertise area
Context Management: Optimal context size for each development task

🚀 Getting Started Today

Quick Start Checklist

✅ Install Core Frameworks

npx bmad-method install
uvx specify init project-name --ai claude

✅ Setup Memory Management

claude mcp add memory-keeper npx mcp-memory-keeper

✅ Configure Shadcn Integration

claude mcp add shadcn-ui npx @shadcn/ui-mcp-server
npx shadcn@latest init

✅ Start Integrated Workflow

# Begin with Spec Kit
/specify [Your project description]

# Move to BMAD planning
*analyst [Analyze specifications]

# Develop with context preservation
*sm [Create memory-aware stories]
*dev [Build with shadcn components]

Pro Tips for Maximum Efficiency

Memory Management Best Practices

Create checkpoints before major feature development
Use descriptive memory keys for easy retrieval
Regularly save architectural decisions and patterns
Set up git-aware context channels for feature branches

Component Development Strategy

Start with shadcn base components
Build custom components following shadcn patterns
Let MCP server handle component installations
Maintain design system consistency through AI agents

Agent Orchestration Tips

Use Analyst for deep requirement understanding
Let Architect make technology stack decisions
Use SM for context-rich story creation
Let Dev focus on implementation with full context
Use QA for maintaining code quality and standards

🎯 Advanced Workflows

Feature Development Cycle

# 1. Specification Phase
/specify Create user profile management with avatar upload, 
personal information editing, and account settings

# 2. Planning Phase  
*analyst --> *pm --> *architect --> *po

# 3. Context Preparation
context_create_checkpoint "profile_system_start"
*po shard documents for profile management feature

# 4. Development Phase
*sm Create profile management stories with shadcn form components
*dev Implement with memory of authentication patterns
*qa Review against established standards

# 5. Memory Preservation
context_add_insight "Profile forms use shadcn form + react-hook-form pattern"
context_save_progress "Profile management MVP completed"

Multi-Feature Coordination

# Feature branch development with isolated context
git checkout -b feature/shopping-cart
context_create_channel "shopping_cart"

# Develop with feature-specific context
*sm Create cart management stories  
*dev Implement cart functionality
context_save_progress "Cart system completed"

# Switch features without context mixing
git checkout feature/payment-system  
context_switch_channel "payment_system"
# Clean context for payment development

This integrated workflow combines the best of all five frameworks, creating a development experience that’s both powerful and maintainable. You get the strategic planning of Spec Kit, the agent specialization of BMAD, the persistent memory of MCP, and the beautiful components of Shadcn - all orchestrated through Claude Code’s powerful AI assistance.

*� The result: Production-ready applications built faster, with better architecture, beautiful UI, and self-correcting code quality - exactly what you need for modern full-stack development!

Ultimate MCP Tools Setup Guide for Claude Code

Mayur Chavhan — Fri, 05 Sep 2025 08:45:00 GMT

Executive Summary

This comprehensive guide provides the complete CLI installation setup for the top 15 MCP tools that will supercharge your Claude Code environment with memory persistence, browser automation, debugging capabilities, and cost-effective API usage. These tools transform Claude Code into a production-ready full-stack development environment.

Top 15 MCP Tools for Full-Stack Development

1. Memory MCP (Context Persistence) ⭐⭐⭐⭐⭐

Purpose: Provides persistent memory across conversations, dramatically reducing API costs by maintaining context

# Install Memory MCP
claude mcp add memory --scope user -- npx -y @mcp-plugins/memory --memory-file ~/claude-memory.json

Features:

Persistent conversation memory across sessions
Entity relationship tracking for complex projects
Cost reduction by avoiding context re-prompting
Structured memory storage in JSON format

Configuration:

{
  "mcpServers": {
    "memory": {
      "command": "npx",
      "args": [
        "@mcp-plugins/memory",
        "--memory-file",
        "/Users/your-username/claude-memory.json"
      ]
    }
  }
}

2. Browser Tools MCP (UI Debugging) ⭐⭐⭐⭐⭐

Purpose: Complete browser automation and debugging solution for UI issues[2]

# Install Browser Tools (requires 2 components)
# Component 1: MCP Server
claude mcp add browser-tools --scope user -- npx -y @agentdeskai/browser-tools-mcp@1.2.1

# Component 2: Middleware Server (run in separate terminal)
npx @agentdeskai/browser-tools-server@1.2.1

Capabilities:

Console log capture and error monitoring
Network request monitoring for API debugging
Automated screenshot capture on errors
Chrome DevTools integration for deep debugging
Lighthouse performance audits

Chrome Extension: Install from BrowserToolsMCP Extension

3. Sequential Thinking MCP (Problem Solving) ⭐⭐⭐⭐⭐

Purpose: Structured problem-solving for complex development challenges[3]

# Install Sequential Thinking
claude mcp add sequential-thinking --scope user -- npx -y @modelcontextprotocol/server-sequential-thinking

Benefits:

Break down complex problems into manageable steps
Revise and refine thoughts as understanding deepens
Branch into alternative reasoning paths
Dynamic thought adjustment based on complexity

4. Filesystem MCP (Local File Access) ⭐⭐⭐⭐⭐

Purpose: Complete filesystem access for reading, writing, and managing project files[4]

# Install Filesystem Access
claude mcp add filesystem --scope user -- npx -y @modelcontextprotocol/server-filesystem ~/Documents ~/Downloads ~/Projects ~/src

Features:

Read/write file operations across specified directories
Directory listing and navigation
File creation and deletion capabilities
Project structure analysis

5. Puppeteer MCP (Browser Automation) ⭐⭐⭐⭐

Purpose: Headless browser automation for testing and debugging

# Install Puppeteer MCP
claude mcp add puppeteer --scope user -- npx -y @modelcontextprotocol/server-puppeteer

Use Cases:

Automated UI testing for full-stack applications
Screenshot generation for documentation
Form interaction testing for authentication flows
Performance monitoring during development

6. Fetch MCP (Web Content Analysis) ⭐⭐⭐⭐

Purpose: Advanced web content fetching and analysis

# Install Fetch MCP
claude mcp add fetch --scope user -- npx -y @kazuhitonakayama/mcp-fetch

Capabilities:

Web page content extraction
Markdown conversion for documentation
API endpoint testing and analysis
Content summarization for research

7. Git MCP (Version Control) ⭐⭐⭐⭐

Purpose: Integrated Git operations within Claude Code

# Install Git MCP
claude mcp add git --scope user -- npx -y @modelcontextprotocol/server-git

Features:

Repository analysis and commit history review
Branch management and merge conflict resolution
Code diff analysis for debugging
Automated commit message generation

8. SQLite MCP (Database Operations) ⭐⭐⭐⭐

Purpose: Local database management and analysis

# Install SQLite MCP
claude mcp add sqlite --scope user -- npx -y @modelcontextprotocol/server-sqlite

Use Cases:

Database schema analysis for debugging
Query optimization and performance tuning
Data migration assistance
Test data generation and seeding

9. GitHub MCP (Repository Integration) ⭐⭐⭐⭐

Purpose: Direct GitHub repository access and management[8]

# Install GitHub MCP
claude mcp add github --scope user -- npx -y @modelcontextprotocol/server-github

Requirements:

GitHub Personal Access Token
Repository permissions configuration

10. Brave Search MCP (Web Research) ⭐⭐⭐

Purpose: Integrated web search for development research

# Install Brave Search (requires API key)
claude mcp add brave-search --scope user -- npx -y @brave/search-mcp

Environment Setup:

export BRAVE_API_KEY="your-brave-api-key"

Advanced Installation & Configuration

Memory MCP Advanced Setup

# Create memory directory
mkdir -p ~/claude-mcp-configs

# Initialize memory file
echo '{
  "entities": [],
  "relations": [],
  "conversations": []
}' > ~/claude-mcp-configs/memory.json

# Install with custom memory file
claude mcp add memory --scope user -- npx -y @mcp-plugins/memory --memory-file ~/claude-mcp-configs/memory.json

Browser Tools Complete Setup

# Step 1: Install Chrome Extension
# Download from: https://github.com/AgentDeskAI/browser-tools-mcp/releases

# Step 2: Start middleware server (keep running)
npx @agentdeskai/browser-tools-server@1.2.1 &

# Step 3: Install MCP server
claude mcp add browser-tools --scope user -- npx -y @agentdeskai/browser-tools-mcp@1.2.1

# Step 4: Open Chrome DevTools and look for "BrowserTools" tab

Filesystem MCP Project-Specific Setup

# Create project structure
mkdir -p ~/Projects/my-fullstack-app/{frontend,backend,database,docs}

# Install with project-specific access
claude mcp add filesystem --scope user -- npx -y @modelcontextprotocol/server-filesystem \
  ~/Projects/my-fullstack-app \
  ~/Documents/Development \
  ~/Downloads

Configuration File Management

Direct Config File Editing (Advanced)

# Open Claude Code config file
code ~/.claude.json

# Or find config location
find ~ -name ".claude.json" 2>/dev/null

Example Complete Configuration:

{
  "mcpServers": {
    "memory": {
      "command": "npx",
      "args": [
        "-y",
        "@mcp-plugins/memory",
        "--memory-file",
        "/Users/your-username/claude-memory.json"
      ]
    },
    "browser-tools": {
      "command": "npx",
      "args": ["-y", "@agentdeskai/browser-tools-mcp@1.2.1"]
    },
    "filesystem": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-filesystem",
        "/Users/your-username/Projects",
        "/Users/your-username/Documents"
      ]
    },
    "sequential-thinking": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-sequential-thinking"]
    },
    "puppeteer": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-puppeteer"]
    }
  }
}

Full-Stack Development Workflow

UI Bug Debugging Workflow

# 1. Start browser tools middleware
npx @agentdeskai/browser-tools-server@1.2.1 &

# 2. Use Claude Code with browser automation
claude "Use browser-tools to:
1. Navigate to localhost:3000
2. Monitor console errors
3. Take screenshots of UI issues
4. Generate bug report with network analysis"

Memory-Enhanced Development

# Use memory for persistent context
claude "Remember that this project uses:
- Next.js 14 with App Router
- Supabase for authentication
- TailwindCSS for styling
- TypeScript throughout

Store this context for future conversations."

Sequential Problem Solving

# Break down complex problems
claude "Use sequential-thinking to analyze this authentication issue:
1. User login flow is broken
2. API returns 401 but token seems valid
3. Need systematic debugging approach"

Cost Optimization Strategies

Memory-Driven Cost Reduction

🧠 Brain Mode: The Memory MCP is your secret weapon for API cost reduction. Instead of re-explaining your entire project context in every conversation, Memory MCP stores and recalls project details, team preferences, and development patterns. This can reduce your token usage by 60-80% in long-term projects.

Setup for Maximum Savings:

# Initialize comprehensive memory
claude "Store in memory:
- Project: Next.js admin dashboard with Supabase
- Tech stack: TypeScript, TailwindCSS, Prisma
- Authentication: Supabase Auth with RLS
- Database: PostgreSQL with migrations
- Deployment: Vercel with preview branches
- Testing: Jest + Playwright for E2E"

Context-Aware Development

# Use filesystem + memory combination
claude "Using filesystem access, analyze my project structure and store the architecture in memory for future reference. Focus on:
1. Component organization patterns
2. API route structure  
3. Database schema relationships
4. Authentication flow implementation"

Essential MCP Tools Installation (One-Command Setup)

Quick Installation Script

#!/bin/bash
# Save as install-mcp-tools.sh and run: chmod +x install-mcp-tools.sh && ./install-mcp-tools.sh

# Memory & Context Tools
claude mcp add memory --scope user -- npx -y @mcp-plugins/memory --memory-file ~/claude-memory.json
claude mcp add sequential-thinking --scope user -- npx -y @modelcontextprotocol/server-sequential-thinking

# Filesystem Access
claude mcp add filesystem --scope user -- npx -y @modelcontextprotocol/server-filesystem ~/Documents ~/Downloads ~/Projects

# Browser Automation & Debugging
claude mcp add browser-tools --scope user -- npx -y @agentdeskai/browser-tools-mcp@1.2.1
claude mcp add puppeteer --scope user -- npx -y @modelcontextprotocol/server-puppeteer

# Web Fetching & Analysis
claude mcp add fetch --scope user -- npx -y @kazuhitonakayama/mcp-fetch
claude mcp add web-search --scope user -- npx -y @brave/search-mcp

# Development Tools
claude mcp add git --scope user -- npx -y @modelcontextprotocol/server-git
claude mcp add sqlite --scope user -- npx -y @modelcontextprotocol/server-sqlite

# Check installation
claude mcp list

Troubleshooting & Verification

Verify Installation

# Check all installed MCP servers
claude mcp list

# Test specific server
claude doctor

# Restart Claude Code if needed
claude restart

Common Issues & Solutions

Server Not Starting:

# Check Node.js version
node --version

# Clear npm cache
npm cache clean --force

# Reinstall problematic server
claude mcp remove server-name
claude mcp add server-name --scope user -- npx -y package-name

Permission Issues:

# Fix filesystem permissions
chmod -R 755 ~/Projects
chmod -R 755 ~/Documents

# Reset Claude config
rm ~/.claude.json
claude mcp add filesystem --scope user -- npx -y @modelcontextprotocol/server-filesystem ~/Projects

Testing Your Setup

# Test browser automation
claude "Use browser-tools to take a screenshot of google.com and analyze the page structure"

# Test memory persistence
claude "What do you remember about my current project?"

# Test filesystem access
claude "List the contents of my Projects directory and analyze the structure"

# Test sequential thinking
claude "Use sequential-thinking to plan the implementation of a new authentication feature"

Production-Ready Workflow Integration

CI/CD Integration

# Create automation script
cat > ~/.claude-automation.sh << 'EOF'
#!/bin/bash
# Start all MCP services
npx @agentdeskai/browser-tools-server@1.2.1 &
echo "Browser tools server started"

# Run automated tests with Claude
claude "Use browser-tools and puppeteer to:
1. Run smoke tests on staging environment
2. Generate performance report
3. Check for console errors
4. Store results in memory for tracking"
EOF

chmod +x ~/.claude-automation.sh

Project-Specific MCP Profiles

# Create project-specific config
mkdir -p ~/Projects/my-app/.claude
cat > ~/Projects/my-app/.claude/config.json << 'EOF'
{
  "mcpServers": {
    "project-filesystem": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-filesystem",
        "/Users/your-username/Projects/my-app"
      ]
    }
  }
}
EOF

This comprehensive setup transforms Claude Code into a production-ready development environment that significantly outperforms basic setups by providing persistent memory, advanced browser automation, comprehensive debugging tools, and substantial API cost savings through intelligent context management.

Complete Guide: Building Beautiful Websites with Claude Code, VSCode, and Design Tools

Mayur Chavhan — Thu, 21 Aug 2025 06:08:00 GMT

Introduction
Prerequisites & Setup
Method 1: SuperDesign - AI Design Agent
Method 2: ShadCN UI Components
Method 3: Website Cloning with FireCrawl
Bonus: Figma to Code Conversion
Best Practices & Tips
Troubleshooting

Introduction

This guide demonstrates three powerful methods to build stunning websites using Claude Code with VSCode. These approaches leverage AI design agents, component libraries, and web scraping to create professional, responsive interfaces with minimal effort.

🧠 The key insight is that generic prompts like “build a beautiful website” produce generic results. Instead, we’ll use specialized tools and iterative workflows to achieve professional-grade designs.

Prerequisites & Setup

System Requirements

Operating System: macOS 10.15+, Windows 10+, or Ubuntu 20.04+
Hardware: 4GB+ RAM (16GB recommended)
Software: Node.js 18+ and VSCode
Network: Internet connection for authentication

Step 1: Install Claude Code

Option A: NPM Installation (Recommended)

npm install -g @anthropic-ai/claude-code

Option B: Native Installation

# macOS/Linux/WSL
curl -fsSL https://claude.ai/install.sh | bash

# Windows PowerShell
irm https://claude.ai/install.ps1 | iex

Step 2: Authentication

Navigate to your project directory and start Claude Code:

cd your-project-directory
claude

Follow the OAuth process to connect your Anthropic account or Claude subscription.

Step 3: Verify Installation

claude doctor

Method 1: SuperDesign - AI Design Agent

SuperDesign is an open-source design agent that works as a VSCode extension, optimized for creating frontend components and UI mockups.

Installation

Install SuperDesign Extension
- Open VSCode Extensions marketplace (Cmd/Ctrl + Shift + X)
- Search for “SuperDesign”
- Click Install

Initialize SuperDesign

# Open command palette
Cmd/Ctrl + Shift + P

# Type and select
SuperDesign: Initialize

Open Design Canvas

# Command palette
Cmd/Ctrl + Shift + P

# Select
SuperDesign: Open Canvas

Workflow Process

Phase 1: Layout Design

Start Claude Code in your project directory and begin with layout iterations:

claude

Example Prompt:

Create 5 different layout iterations for an Uber-style ride-sharing app home screen. Focus on the main structure and component placement.

SuperDesign will generate ASCII layouts in the terminal, allowing you to quickly evaluate different structures before writing code.

Phase 2: Theme Development

Select your preferred layout and move to styling:

Example Prompt:

I like layout #2. Now create 5 different theme variations focusing on:
- Color palette application
- Typography choices
- Visual hierarchy
- Design language consistency

Phase 3: Color Palette Integration

Use Coolors.co to generate color palettes:

Visit coolors.co
Press spacebar to generate palettes
Copy the CSS values
Provide to Claude Code:

Apply this color palette to the design:
--primary: #FF6B6B;
--secondary: #4ECDC4;
--accent: #45B7D1;
--neutral: #96CEB4;
--background: #FFEAA7;

Phase 4: Animation & Interactivity

Add micro-interactions and animations:

Add subtle animations to this design:
- Hover effects for buttons
- Loading states
- Smooth transitions
- Interactive feedback for user actions

Advanced Animation Resources

Visit Animatopy for curated animation effects:

Browse animation collections
Copy HTML/CSS code
Integrate into your design through Claude Code

SuperDesign Canvas Features

Real-time Preview: See changes instantly
Device Testing: Switch between desktop, tablet, mobile views
Interactive Elements: Click buttons, type in inputs, test functionality
Export Options: Copy designs or generate code for other frameworks

Method 2: ShadCN UI Components

Build with professionally designed, accessible components using the ShadCN MCP server.

Setup ShadCN MCP Server

Installation Command:

claude mcp add-json "shadcn-ui-server" '{"command":"npx","args":["-y","shadcn-ui-mcp-server"]}'

Create Workflow Commands

Create a custom slash command for consistent ShadCN workflows. In your project, create:

File: commands/shadcn.md

# ShadCN Development Workflow

## Planning Phase
1. Always use the ShadCN MCP server for component context
2. Plan the entire application structure before implementation
3. List all required components and their relationships
4. Create an implementation plan in implementation.md

## Implementation Phase  
1. Reference the implementation plan
2. Use ShadCN MCP server for all component implementations
3. Ensure proper component structure and styling
4. Implement responsive design patterns
5. Add proper TypeScript types

Usage Workflow

Step 1: Planning

/shadcn Plan an e-commerce dashboard using ShadCN components. Create a detailed implementation plan in implementation.md including:
- Component hierarchy
- Required ShadCN components
- Layout structure
- Responsive considerations

Step 2: Implementation

/shadcn Implement the dashboard based on the implementation plan in implementation.md. Use the ShadCN MCP server for accurate component usage.

Theme Customization with TweakCN

Visit TweakCN.com
Choose or customize a theme
Copy the CSS variables
Apply to your project:

Apply this TweakCN theme to my ShadCN components:

:root {
  --background: 0 0% 100%;
  --foreground: 222.2 84% 4.9%;
  --card: 0 0% 100%;
  --card-foreground: 222.2 84% 4.9%;
  /* ... rest of theme variables */
}

Method 3: Website Cloning with FireCrawl

Clone existing websites and adapt them with your content using the FireCrawl MCP server.

Setup FireCrawl MCP

Installation:

claude mcp add-json "firecrawl" '{"command":"mcp-server-firecrawl","env":{"FIRECRAWL_API_KEY":"your-api-key"}}'

Get your API key from Firecrawl.dev

Cloning Workflow

Basic Website Clone

Create a 1:1 clone of https://example-website.com including:
- Layout structure
- Component hierarchy  
- Styling patterns
- Interactive elements

Enhanced Clone with Screenshots

For better results, include screenshots in your prompt:

Clone this website: https://example-website.com

[Include screenshot of the target website]

Focus on:
- Exact visual hierarchy
- Color scheme accuracy
- Typography matching
- Responsive behavior

Customization After Cloning

Adapt the cloned design with my brand:
- Replace content with [your content]
- Update color scheme to [your colors]
- Modify typography to [your fonts]
- Add/remove sections as needed

Bonus: Figma to Code Conversion

Convert Figma designs directly to code using the Figma MCP server.

Setup Methods

Option A: Composio Figma MCP

# Generate command at mcp.composio.dev
npx @composio/mcp@latest setup "https://mcp.composio.dev/partner/composio/figma/mcp?customerId=YOUR_ID" "figma-server-name" --client

Option B: Native Figma MCP (Requires Figma Desktop)

claude mcp add --transport sse figma-dev-mode-mcp-server http://127.0.0.1:3845/sse

Usage Patterns

Selection-Based Conversion

Select a frame/component in Figma
In Claude Code:

Convert my current Figma selection to React components with Tailwind CSS

Link-Based Conversion

Convert this Figma design to code: [figma-share-link]

Requirements:
- Use React/Next.js
- Implement responsive design
- Match colors and typography exactly
- Include hover states and interactions

Best Practices & Tips

Iterative Development Approach

🧠 The power of Claude Code lies in continuous iteration. Unlike human developers, AI agents never tire of refinement, making them ideal for polishing designs through multiple cycles.

Start with HTML files during initial iterations (easier to modify than full frameworks)
Use multiple iterations (5+ variations) for each design phase
Break down complex requests into smaller, manageable tasks
Leverage real-time preview to validate changes immediately

Design System Consistency

Establish design tokens early (colors, typography, spacing)
Create reusable components before building pages
Maintain style guides throughout development
Test responsive behavior across devices regularly

Performance Considerations

Optimize this design for performance:
- Minimize bundle size
- Implement lazy loading
- Optimize images and assets
- Ensure accessibility standards

Quality Assurance

Review this implementation for:
- Accessibility compliance (WCAG 2.1)
- Cross-browser compatibility
- Mobile responsiveness  
- Performance optimization
- Code maintainability

Troubleshooting

Common Installation Issues

Permission Errors with NPM

# Don't use sudo - configure npm properly instead
mkdir -p ~/.npm-global
npm config set prefix ~/.npm-global
echo 'export PATH=~/.npm-global/bin:$PATH' >> ~/.bashrc
source ~/.bashrc

MCP Server Connection Issues

# Check MCP status
/mcp

# Restart Claude Code if servers aren't connecting
claude restart

SuperDesign Canvas Not Loading

Ensure VSCode is updated to latest version
Restart VSCode after installing extension
Check that SuperDesign is initialized:

Cmd/Ctrl + Shift + P > SuperDesign: Initialize

Design Quality Issues

Generic-Looking Results

Problem: Using vague prompts like “make it beautiful”
Solution: Provide specific design requirements, references, and constraints

Inconsistent Styling

Problem: Not establishing design system early
Solution: Define color palettes, typography, and component patterns before implementation

Poor Responsive Behavior

Problem: Desktop-first approach without mobile consideration
Solution: Test designs across device sizes throughout development

Performance Problems

Slow Loading Times

Analyze this codebase for performance bottlenecks and suggest optimizations for:
- Bundle size reduction
- Asset optimization
- Lazy loading implementation
- Critical rendering path

Browser Compatibility Issues

Ensure this code works across modern browsers:
- Add necessary polyfills
- Use progressive enhancement
- Implement fallbacks for newer CSS features
- Test in Chrome, Firefox, Safari, Edge

Advanced Techniques

Multi-Tool Workflows

Combine multiple tools for maximum effectiveness:

SuperDesign for initial layout and theming
ShadCN MCP for component implementation
FireCrawl for reference and inspiration
Figma MCP for design system integration

Custom Workflow Automation

Create project-specific commands:

File: commands/design-system.md

# Custom Design System Workflow

When building with our design system:
1. Use these color variables: [your colors]
2. Apply these typography scales: [your fonts]
3. Follow these spacing patterns: [your spacing]
4. Implement these animation principles: [your animations]

Framework Integration

Convert this design to:
- Next.js with App Router
- React with TypeScript
- Vue.js with Composition API
- Svelte with SvelteKit
- Pure HTML/CSS/JavaScript

🧠 Remember: The best AI-generated designs come from clear requirements, iterative refinement, and leveraging the right tools for each phase of development. Start with one method that matches your project needs, then gradually incorporate other techniques as you become more comfortable with the workflow.

Amazon Q Developer CLI: Complete Guide for DevOps and Development Teams

Mayur Chavhan — Fri, 20 Jun 2025 18:52:00 GMT

1. Introduction & Benefits Analysis

What is Amazon Q Developer CLI

Amazon Q Developer CLI is a generative AI-powered conversational assistant that revolutionizes command-line development workflows. Unlike traditional AWS CLI tools, Amazon Q Developer CLI combines natural language processing with direct access to AWS services, enabling developers to interact with their infrastructure through conversational interfaces. The tool integrates contextual information from your local development environment, providing enhanced understanding of your specific use case and delivering relevant, context-aware responses.

Key Differences from Other AWS CLI Tools

🧠 Brain Mode: Key Differentiators

Amazon Q Developer CLI stands apart from traditional AWS CLI tools in several fundamental ways. While the standard AWS CLI requires memorizing specific command syntax and parameters, Amazon Q Developer CLI accepts natural language instructions and translates them into executable commands. This represents a paradigm shift from command-based interfaces to conversation-based development workflows.

The tool’s AI agent capabilities allow it to understand context, maintain conversation history, and provide iterative improvements based on feedback. Unlike static CLI tools that execute single commands, Amazon Q Developer CLI can perform multi-step operations, analyze outputs, and make intelligent decisions about next steps.

Amazon Q Developer CLI supports hundreds of popular command-line tools including git, npm, docker, and aws, providing IDE-style autocompletion and contextual suggestions. The enhanced CLI agent, powered by Claude 3.7 Sonnet, can read and write files locally, query AWS resources, and create code iteratively based on user feedback.

Integration Capabilities with Existing AWS Toolchain

The CLI integrates seamlessly with existing AWS profiles and credentials, supporting both AWS Builder ID authentication for free tier usage and IAM Identity Center integration for enterprise Pro tier subscriptions. It works alongside traditional AWS CLI installations, respecting existing profile configurations and credential management.

Amazon Q Developer CLI features Console-to-Code functionality that records AWS Management Console actions and generates corresponding CLI commands or Infrastructure as Code templates in multiple formats including CDK Java, Python, TypeScript, and CloudFormation JSON/YAML. This bridges the gap between console-based prototyping and production-ready automation code.

Performance Improvements and Productivity Gains

Internal Amazon studies demonstrate significant productivity improvements with Amazon Q Developer CLI usage:

80% faster development tasks across common software development workflows
40% boost in developer productivity through AI-assisted code generation and debugging
Amazon engineers upgraded 50% of their production Java systems using the tool, saving an estimated 4,500 developer-years of effort and $260M in annualized efficiency gains

The tool optimizes AWS operations and cloud cost efficiency by providing intelligent recommendations for resource utilization and suggesting cost-effective alternatives during infrastructure provisioning.

2. Installation & Setup Guide

macOS Installation

Method 1: Direct Download

Download the macOS installer (.dmg file) from the official AWS documentation page
Open the downloaded .dmg file
Drag the Amazon Q app into your Applications folder
Launch the application and follow the setup wizard

Method 2: Homebrew Installation

brew install amazon-q

Post-Installation Setup for macOS:

Enable shell integrations when prompted
Grant accessibility permissions in System Settings → Privacy & Security → Accessibility
Complete authentication setup using your AWS Builder ID or IAM Identity Center credentials

Linux Installation

Ubuntu/Debian Package Installation:

# Update package list
sudo apt update

# Install required dependencies
sudo apt install libfuse2 curl unzip

# Download the Debian package
curl --proto '=https' --tlsv1.2 -sSf https://desktop-release.q.us-east-1.amazonaws.com/latest/amazon-q.deb -o amazon-q.deb

# Install the package
sudo apt install -y ./amazon-q.deb

AppImage Installation (GUI Required):

# Download AppImage
curl --proto '=https' --tlsv1.2 -sSf https://desktop-release.q.us-east-1.amazonaws.com/latest/amazon-q.appimage -o amazon-q.appimage

# Make executable
chmod +x amazon-q.appimage

# Run the AppImage
./amazon-q.appimage

Zip File Installation (Headless Environments):

# Download installer
curl --proto '=https' --tlsv1.2 -sSf "https://desktop-release.q.us-east-1.amazonaws.com/latest/q-x86_64-linux.zip" -o "q.zip"

# Extract and install
unzip q.zip
cd q
chmod +x install.sh
./install.sh

# Source your shell configuration or restart terminal
source ~/.bashrc  # or ~/.zshrc for zsh users

Windows Installation (via WSL)

Amazon Q Developer CLI does not have native Windows support as of June 2025, requiring Windows Subsystem for Linux (WSL) for installation.

Step 1: Install WSL

wsl --install

Step 2: Launch Ubuntu Environment

wsl -d Ubuntu

Step 3: Install Amazon Q Developer CLI in WSL

# Navigate to home directory
cd
pwd  # Should show /home/your-username

# Download installer
curl --proto '=https' --tlsv1.2 -sSf https://desktop-release.codewhisperer.us-east-1.amazonaws.com/latest/q-x86_64-linux-musl.zip -o q.zip

# Install unzip if needed
sudo apt install unzip

# Extract and install
unzip q.zip
cd q
chmod +x install.sh
./install.sh

# Restart shell
bash

Authentication Setup and Configuration

AWS Builder ID Authentication (Free Tier):

q login

Select “Use for Free with Builder ID” when prompted. This opens a browser window for authentication. The Builder ID provides access to free tier features without requiring an AWS account.

IAM Identity Center Authentication (Pro Tier): For Pro tier subscriptions, organizations must configure IAM Identity Center and create appropriate permission sets. Users authenticate using their organization’s start URL provided by AWS administrators.

Initial Configuration and Profile Management

Verify Installation:

q --version
q doctor  # Diagnoses configuration issues

Configure Editor Preference:

# Set preferred editor for multi-line prompts
export EDITOR=nano  # or vim, code, etc.

# Make permanent by adding to shell config
echo 'export EDITOR=nano' >> ~/.bashrc

Integration Settings:

# Enable/disable inline completions
q inline enable
q inline disable

# Configure MCP server timeout
q settings mcp.initTimeout 5000  # 5 seconds

Integration with Existing AWS CLI Profiles

Amazon Q Developer CLI respects existing AWS CLI profiles and credentials. It uses the same credential resolution order as the standard AWS CLI:

Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
AWS credentials file (~/.aws/credentials)
AWS config file (~/.aws/config)
IAM roles for EC2 instances
IAM roles for ECS tasks
IAM roles for Lambda functions

# Use specific AWS profile
export AWS_PROFILE=my-profile
q chat "List S3 buckets in us-west-2"

# Or specify profile in conversation
q chat "Using the production profile, show me EC2 instances"

3. AWS Infrastructure Automation

Practical Examples for Common AWS Infrastructure Tasks

EC2 Instance Management:

q chat "Launch a new t3.micro EC2 instance in us-east-1 with Amazon Linux 2"

Amazon Q Developer CLI will generate and execute the appropriate AWS CLI commands, handling subnet selection, security group configuration, and key pair management based on your account’s existing resources.

S3 Bucket Operations:

q chat "Create an S3 bucket with versioning enabled and set up lifecycle policies to transition objects to IA after 30 days"

RDS Database Deployment:

q chat "Deploy a MySQL RDS instance with Multi-AZ enabled and automated backups configured for 7-day retention"

Infrastructure as Code (IaC) Integration

CloudFormation Integration: Amazon Q Developer CLI can generate CloudFormation templates based on natural language descriptions:

q chat "Generate a CloudFormation template for a three-tier web application with ALB, Auto Scaling Group, and RDS"

The tool generates both JSON and YAML CloudFormation templates, including:

Resource definitions with appropriate properties
Parameter sections for customization
Output sections for cross-stack references
Proper dependency management

CDK Integration:

q chat "Create a CDK Python stack for a serverless application with API Gateway, Lambda, and DynamoDB"

Generates CDK code in multiple languages:

TypeScript
Python
Java
C#/.NET

Console-to-Code Workflow:

Perform actions in AWS Management Console
Use Console-to-Code to record actions
Generate equivalent CLI commands or IaC templates:

# After recording console actions
q chat "Convert my recorded console actions to Terraform configuration"

Automated Deployment Pipelines and CI/CD Integration

GitHub Actions Integration:

name: Deploy with Amazon Q
on: [push]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Setup Amazon Q CLI
        run: |
          curl -sSf https://desktop-release.q.us-east-1.amazonaws.com/latest/q-x86_64-linux.zip -o q.zip
          unzip q.zip && ./q/install.sh
      - name: Deploy Infrastructure
        run: |
          q chat "Deploy the infrastructure defined in ./infra/ directory"

AWS CodePipeline Integration: Amazon Q Developer CLI can generate CodePipeline configurations and integrate with existing CI/CD workflows:

q chat "Create a CodePipeline that builds my Node.js app from GitHub and deploys to ECS"

Resource Provisioning and Management Workflows

Multi-Environment Management:

# Development environment setup
q chat "Create a development environment with smaller instance types and single-AZ deployment"

# Production environment setup
q chat "Create a production environment with high availability, auto-scaling, and monitoring"

Resource Tagging and Organization:

q chat "Apply consistent tagging strategy across all resources with Environment, Project, and Owner tags"

Cost Optimization Automation Strategies

Right-Sizing Recommendations:

q chat "Analyze my EC2 instances and recommend right-sizing opportunities"

Reserved Instance Analysis:

q chat "Analyze my usage patterns and recommend Reserved Instance purchases"

Automated Cleanup:

q chat "Identify and clean up unused EBS volumes, unattached Elastic IPs, and outdated AMIs"

4. Application Development Features

Code Generation and Scaffolding Capabilities

Amazon Q Developer CLI excels at generating complete application scaffolds and boilerplate code. The enhanced CLI agent can create entire project structures based on natural language descriptions:

Web Application Scaffolding:

q chat "Build a simple fact checking app"

This command generates:

Complete project structure with appropriate directories
Package.json with necessary dependencies
Basic HTML templates and CSS styling
JavaScript/Python backend code with API endpoints
Database schema and migration files
README documentation with setup instructions

Microservice Architecture Generation:

q chat "Create a microservices architecture with user authentication, product catalog, and order processing services using Node.js and Docker"

Integration with Popular IDEs and Development Environments

VS Code Integration: While Amazon Q Developer CLI operates from the terminal, it integrates seamlessly with IDE workflows:

# Open current project in VS Code and start Q chat
code .
q chat "Analyze this codebase and suggest architectural improvements"

Terminal-Based Development:

# Multi-line prompt composition using /editor command
q chat
Amazon Q> /editor
# Opens your preferred editor for complex prompts

SSH Integration for Remote Development:

# Install SSH integration for remote servers
q integrations install ssh

This enables Amazon Q Developer CLI functionality when working on remote servers via SSH.

Debugging and Troubleshooting Assistance

Error Analysis and Resolution:

q chat "My Express.js application is throwing 'Cannot read property of undefined' errors. Help me debug this issue."

Amazon Q analyzes error messages, examines code context, and provides:

Root cause identification
Step-by-step debugging instructions
Code fixes with explanations
Prevention strategies for similar issues

Log Analysis:

q chat "Analyze these CloudWatch logs and identify the cause of high response times"

Performance Troubleshooting:

q chat "My Lambda function is experiencing cold start issues. Optimize it for better performance."

Code Review and Optimization Suggestions

Automated Code Review:

# Select code and request review
q chat "Review this Python function for security vulnerabilities and performance improvements"

Code Quality Enhancement:

q chat "Refactor this code to follow clean code principles and improve maintainability"

Security Analysis:

q chat "Scan this code for security vulnerabilities and suggest fixes"

Testing Automation and Quality Assurance Features

Unit Test Generation:

q chat "Generate comprehensive unit tests for this user authentication module"

Integration Test Creation:

q chat "Create integration tests for my REST API endpoints with different scenarios"

Test-Driven Development Support:

q chat "Help me write tests first for a shopping cart feature, then implement the functionality"

5. AWS Infrastructure Visualization

Creating Architecture Diagrams and Infrastructure Graphs

Amazon Q Developer CLI can generate AWS architecture diagrams when integrated with MCP (Model Context Protocol) servers. This powerful combination enables automated diagram creation from natural language descriptions:

Setup MCP for Architecture Diagrams:

# Install uv for MCP server management
sudo snap install astral-uv --classic

# Create MCP configuration
mkdir -p ~/.aws/amazonq
cat > ~/.aws/amazonq/mcp.json  /compact

Editor Integration for Complex Prompts:

# Use /editor for multi-line prompts
Amazon Q> /editor

# Set permanent editor preference
export EDITOR=code  # or vim, nano, etc.
echo 'export EDITOR=code' >> ~/.bashrc

Scripting and Automation Workflows

Automated Deployment Scripts:

#!/bin/bash
# deploy.sh - Automated deployment script

# Start Q session and deploy
q chat --no-interactive --trust-all-tools "
Deploy the application in ./src directory to AWS:
1. Build the Docker image
2. Push to ECR
3. Update ECS service
4. Verify deployment health
"

Infrastructure Provisioning Automation:

# infrastructure-setup.sh
q chat --no-interactive "
Create production infrastructure:
- VPC with public/private subnets
- Application Load Balancer
- ECS cluster with Fargate
- RDS MySQL with Multi-AZ
- CloudWatch logging and monitoring
"

Integration with Other AWS Services and Third-Party Tools

GitHub Integration: Amazon Q Developer CLI provides direct GitHub integration for automated workflows:

q chat "Review this pull request and suggest improvements"
q chat "Create a new feature branch and implement user authentication"

Docker Integration:

q chat "Containerize my Python application and create optimal Dockerfile"
q chat "Set up multi-stage Docker build for production deployment"

Terraform Integration:

q chat "Convert my CloudFormation templates to Terraform configuration"
q chat "Generate Terraform modules for reusable infrastructure components"

Security Best Practices and Access Management

IAM Policy Management:

q chat "Review my IAM policies for least privilege compliance"
q chat "Create role-based access control for my development team"

Security Group Optimization:

q chat "Audit my security groups and recommend improvements"
q chat "Implement network segmentation best practices"

Credential Management:

# Use IAM roles instead of long-term credentials
q chat "Help me configure IAM roles for EC2 instances instead of using access keys"

Performance Optimization Techniques

Query Optimization:

Use specific, detailed prompts rather than vague requests
Provide context about your environment and goals
Break complex tasks into smaller, focused questions

Resource Efficiency:

# Efficient resource usage
q chat --trust-tools=fs_read,fs_write "Optimize my code for better performance"

Conversation Management:

# Start fresh conversations for different topics
Amazon Q> /clear

# Use context effectively
Amazon Q> /context path/to/relevant/files

7. Pricing & Limitations Analysis

Detailed Comparison of Free Tier vs. Paid Plans

Amazon Q Developer Free Tier (Free):

Code Suggestions: Unlimited in IDE and CLI
CLI Completions: Free for public CLI tools
Chat Interactions: 50 interactions per month in IDE
Software Development Agents: 10 invocations per month
Code Transformation: 1,000 lines of code per month
AWS Account Queries: 25 queries per month
Console Error Diagnosis: Included
License Reference Tracking: Included

Amazon Q Developer Pro Tier ($19/month per user):

All Free Tier Features: Plus enhanced capabilities
Chat Interactions: Unlimited in IDE
Software Development Agents: Unlimited invocations
Code Transformation: 4,000 lines of code per month
AWS Account Queries: Unlimited
Generative SQL: 1,000 queries per month
Enterprise Access Controls: IAM Identity Center integration
Customization: Adapt to your codebase for better suggestions
Additional Charge: $0.003 per line for Java transformation beyond monthly limit

Feature Limitations and Usage Quotas

Free Tier Limitations:

Limited monthly interactions may restrict heavy usage
AWS account resource queries capped at 25 per month
Code transformation limited to 1,000 lines monthly
Content may be used for service improvement

Pro Tier Quotas:

Most features have high or unlimited usage limits
Code transformation pooled at account level (4,000 lines/month)
Some internal quotas exist (e.g., 30 software development agent invocations/month)
No direct upgrade path from Builder ID to Pro tier - requires new subscription

Cost-Benefit Analysis for Different Team Sizes

Individual Developers:

Free Tier: Suitable for occasional use, personal projects, learning
Pro Tier ($19/month): Cost-effective for professional developers with regular AI assistance needs

Small Teams (2-10 developers):

Monthly Cost: $38-$190 for Pro tier
ROI Calculation:
- Average developer hourly cost: $50-$100
- Time savings: 5-6 hours per week per developer
- Monthly savings: $1,000-$2,400 per developer
- ROI: 500-1,200% return on investment

Enterprise Teams (50+ developers):

Monthly Cost: $950+ for Pro tier
Benefits:
- Standardized development practices
- Reduced onboarding time for new developers
- Consistent code quality across teams
- Enterprise security and access controls

Migration Path from Free to Paid Plans

Important Migration Considerations:

No direct upgrade path from Builder ID to Pro tier
Users must create new IAM Identity Center accounts for Pro access
Existing conversations and settings don’t transfer
Organizations need IAM Identity Center setup before Pro activation

Migration Steps:

Set up IAM Identity Center in AWS account
Subscribe to Amazon Q Developer Pro in AWS console
Create user groups and assign permissions
Users sign out of Builder ID sessions
Users authenticate with IAM Identity Center credentials

Alternative Tools Comparison

Primary Alternatives:

ChatGPT: General-purpose AI assistant, not AWS-specific
GitHub Copilot: IDE-focused, limited CLI capabilities
Cody (Sourcegraph): Code-centric, enterprise context awareness
Windsurf Editor: IDE with built-in AI assistance

When to Choose Amazon Q Developer CLI:

Heavy AWS infrastructure management
Need for CLI-based AI assistance
Integration with existing AWS toolchain
Cost optimization and AWS best practices guidance
Console-to-Code workflow requirements

8. Real-World Workflow Integration

DevOps Pipeline Integration Examples

CI/CD Pipeline Enhancement:

# In GitHub Actions workflow
- name: Deploy with Amazon Q
  run: |
    q chat --no-interactive --trust-all-tools "
    Deploy application to staging environment:
    1. Run test suite
    2. Build Docker image
    3. Deploy to ECS staging cluster
    4. Run smoke tests
    5. If successful, promote to production
    "

Infrastructure Drift Detection:

# Scheduled drift detection script
#!/bin/bash
q chat --no-interactive "
Check for infrastructure drift:
1. Compare current AWS resources with Terraform state
2. Identify any manual changes
3. Generate drift report
4. Send alert if discrepancies found
"

Developer Workflow Optimization Scenarios

Feature Development Workflow:

# Start new feature
q chat "Create a new feature branch for user authentication and set up basic structure"

# During development
q chat "Review my authentication code for security best practices"

# Pre-commit checks
q chat "Run code quality checks and fix any issues before committing"

Code Review Acceleration:

# Automated code review
q chat "Analyze this pull request for potential issues, security vulnerabilities, and performance improvements"

Team Collaboration Features and Setup

Shared Configuration Management:

// .q-config.json - Team-shared configuration
{
  "profiles": {
    "development": {
      "aws_profile": "dev",
      "default_region": "us-west-2"
    },
    "production": {
      "aws_profile": "prod",
      "default_region": "us-east-1"
    }
  },
  "trusted_tools": ["fs_read", "fs_write", "use_aws"],
  "editor": "code"
}

Team Best Practices:

# Standardized deployment commands
alias deploy-dev="q chat --profile development --trust-all-tools 'Deploy to development environment'"
alias deploy-prod="q chat --profile production 'Deploy to production with approval checks'"

Monitoring and Alerting Integration

Proactive Monitoring Setup:

q chat "Set up comprehensive monitoring for my web application including:
- Application performance metrics
- Infrastructure health checks
- Custom business metrics
- Automated alerting for anomalies"

Alert Response Automation:

# Incident response script
q chat "Investigate the high CPU alert:
1. Check CloudWatch metrics for the affected instances
2. Analyze application logs for errors
3. Suggest immediate mitigation steps
4. Create incident report"

Incident Response and Troubleshooting Workflows

Systematic Troubleshooting Process: Amazon Q Developer CLI provides structured incident response capabilities:

q chat "Our production NGINX application is experiencing 502 Gateway Timeout errors. Help investigate and diagnose the issue."

The tool systematically:

Discovers infrastructure components (ECS clusters, services, tasks)
Checks service health and status
Analyzes logs across multiple services
Identifies root causes through correlation
Provides step-by-step remediation guidance
Validates fixes and confirms resolution

Automated Log Analysis:

q chat "Analyze the last hour of CloudWatch logs to identify the cause of increased error rates"

Performance Issue Resolution:

q chat "My database is experiencing slow query performance. Help identify bottlenecks and optimize queries."

9. Quick Reference & Troubleshooting

Essential Commands Reference

| Command | Description | Example | | --- | --- | --- | | q login | Authenticate with Builder ID or IAM Identity Center | q login | | q chat | Start interactive chat session | q chat "help with deployment" | | q chat --resume | Resume previous conversation | q chat --resume | | q --version | Show CLI version | q --version | | q doctor | Diagnose configuration issues | q doctor | | q inline enable/disable | Toggle autocomplete | q inline disable | | /editor | Open text editor for complex prompts | /editor | | /clear | Start new conversation | /clear | | /compact | Get concise responses | /compact | | /context | Add files to conversation | /context path/to/file |

Common Troubleshooting Issues

Installation Issues:

Terminal not detecting q command

# Restart terminal or source shell config
source ~/.bashrc  # or ~/.zshrc

Permission denied errors on Linux

chmod +x install.sh
./install.sh

Authentication Issues:

Bearer token refresh errors

rm ~/.aws/qcodetransform/credentials.json
qct transform  # Re-authenticate

Builder ID to Pro tier transition

Cannot directly upgrade from Builder ID to Pro tier
Must create new IAM Identity Center account
Sign out of existing session before authenticating with Pro credentials

Performance Issues:

Slow response times

Use more specific prompts
Break complex requests into smaller tasks
Check internet connectivity and AWS service status

CLI integration not working after autostart

Manually restart the Amazon Q CLI service
Check accessibility permissions on macOS
Verify shell integration installation

Links to Official Documentation

The Complete Beginner's Guide to uv: Python's Lightning-Fast Package Manager

Mayur Chavhan — Tue, 03 Jun 2025 13:53:00 GMT

uv is a modern, high-performance Python package manager written in Rust that serves as a drop-in replacement for traditional tools like pip, pip-tools, virtualenv, poetry, and more. With speeds up to 100x faster than pip and a unified approach to Python project management, uv represents the next generation of Python tooling.

🧠 Why uv Matters: Unlike traditional Python package managers that evolved separately over time, uv was designed from the ground up as a unified solution. It combines package installation, dependency resolution, virtual environment management, Python version handling, and project scaffolding into a single, lightning-fast tool. This means fewer commands to remember, faster installations, and a more consistent development experience.

What Makes uv Special

uv offers several compelling advantages over existing tools:

Extreme Speed: 10-100x faster than pip, with installations completing in seconds rather than minutes
All-in-One Solution: Replaces pip, pip-tools, virtualenv, poetry, pyenv, and more
Drop-in Compatibility: Works with existing pip workflows and requirements.txt files
Modern Architecture: Built in Rust for optimal performance and reliability
Automatic Environment Management: Handles virtual environments seamlessly behind the scenes

Installation

Getting started with uv is straightforward, with multiple installation options to suit different preferences:

Quick Installation

For Unix/Linux/macOS:

curl -LsSf https://astral.sh/uv/install.sh | sh

For Windows:

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Alternative Installation Methods

Via Homebrew (macOS):

brew install uv

Via pip (if you have Python already):

pip install uv

Verify Installation:

uv --version
# Output: uv 0.7.9 (or latest version)

Core Project Management

Creating a New Python Project from Scratch

🧠 Project Creation Philosophy: uv takes an opinionated approach to project structure, automatically setting up best practices that many developers configure manually. This includes proper directory structure, configuration files, and even Git initialization.

Basic Project Creation

# Create a simple application project
uv init my-awesome-app
cd my-awesome-app

# View the created structure
ls -la
# Output:
# .python-version
# README.md
# main.py
# pyproject.toml

Project Type Options

# For installable packages
uv init my-package --package

# For libraries (with src/ layout)
uv init my-library --lib

# For applications (with app/ layout)
uv init my-web-app --app

# Minimal setup without sample files
uv init my-project --bare

Expected Output:

Initialized project `my-awesome-app` at `/path/to/my-awesome-app`

Initializing uv in an Existing Python Project

If you have an existing Python project, you can easily add uv support:

# Navigate to your existing project
cd /path/to/existing-project

# Initialize uv (won't overwrite existing files)
uv init --bare

# Import existing requirements
uv add -r requirements.txt
uv add -r requirements-dev.txt --dev

🧠 Compatibility Note: uv is designed to work alongside existing Python projects. The --bare flag ensures that uv won’t overwrite your existing main.py or README.md files, making migration safe and non-destructive.

Project Structure and Configuration Files

Key Files Explained

pyproject.toml - The central configuration file:

[project]
name = "my-awesome-app"
version = "0.1.0"
description = "My awesome Python application"
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
    "requests>=2.28.0",
    "fastapi>=0.95.0",
]

[tool.uv]
dev-dependencies = [
    "pytest>=7.4.0",
    "black>=23.7.0",
    "ruff>=0.0.280",
]

.python-version - Specifies the Python version for the project:

3.11

uv.lock - Automatically generated lockfile containing exact dependency versions (similar to package-lock.json in Node.js):

version = 1

[[package]]
name = "requests"
version = "2.30.0"
dependencies = ["certifi>=2017.4.17", "charset-normalizer>=2.0.0"]

🧠 Lockfile Benefits: The uv.lock file ensures that everyone on your team gets exactly the same versions of dependencies, preventing the “it works on my machine” problem. Unlike pip’s requirements.txt, uv.lock includes transitive dependencies and cross-platform compatibility information.

Package Management

Adding, Removing, and Updating Packages

Adding Dependencies

# Add a basic package
uv add requests
# Output: Resolved 4 packages in 234ms
#         Installed 4 packages in 12ms
#         + certifi==2024.7.4
#         + charset-normalizer==3.3.2
#         + idna==3.7
#         + requests==2.32.3

# Add package with version constraint
uv add "django>=4.2.0,<5.0.0"

# Add package with extras
uv add "fastapi[all]"

# Add multiple packages at once
uv add requests flask sqlalchemy

Removing Dependencies

# Remove a package
uv remove requests
# Output: Removed 4 packages in 5ms
#         - certifi==2024.7.4
#         - charset-normalizer==3.3.2
#         - idna==3.7
#         - requests==2.32.3

# Remove multiple packages
uv remove flask sqlalchemy

Updating Dependencies

# Update all dependencies
uv lock --upgrade

# Update specific package
uv lock --upgrade-package requests

# Update to latest compatible versions
uv add requests --upgrade

Development vs Production Dependencies

uv provides clear separation between production and development dependencies:

# Add production dependencies
uv add fastapi uvicorn

# Add development dependencies
uv add --dev pytest black ruff mypy

# Add to specific dependency groups
uv add --group test pytest pytest-cov
uv add --group lint black ruff

pyproject.toml structure:

[project]
dependencies = [
    "fastapi>=0.100.0",
    "uvicorn>=0.20.0",
]

[tool.uv.dev-dependencies]
test = [
    "pytest>=7.4.0",
    "pytest-cov>=4.1.0",
]
lint = [
    "black>=23.7.0",
    "ruff>=0.0.280",
]

Version Constraints and Lock Files

Version Constraint Examples

# Exact version
uv add "django==4.2.7"

# Minimum version
uv add "requests>=2.28.0"

# Version range
uv add "flask>=2.0.0,<3.0.0"

# Compatible release
uv add "numpy~=1.24.0"  # Equivalent to >=1.24.0,<1.25.0

# Pre-release versions
uv add --prerelease=allow "django>=5.0.0"

Working with Lock Files

# Generate/update lock file
uv lock

# Install from lock file
uv sync

# Install only production dependencies
uv sync --no-dev

# Install with specific groups
uv sync --group test --group lint

🧠 Lock File Strategy: Always commit uv.lock to version control. This ensures that your CI/CD pipeline, production deployments, and teammate environments use exactly the same dependency versions, eliminating dependency-related bugs.

Installing from Different Sources

PyPI (Default)

uv add requests  # Latest version from PyPI

Git Repositories

# From GitHub
uv add git+https://github.com/django/django.git

# Specific branch
uv add git+https://github.com/django/django.git@main

# Specific tag
uv add git+https://github.com/django/django.git@4.2.7

# Specific commit
uv add git+https://github.com/django/django.git@abc123def456

Local Paths

# Local package in development
uv add ./my-local-package

# Editable install (changes reflected immediately)
uv add -e ./my-local-package

# From local wheel file
uv add ./dist/my-package-1.0.0-py3-none-any.whl

Private Package Indexes

# Custom index
uv add --index https://my-private-pypi.com/simple/ my-private-package

# With authentication
UV_INDEX_USERNAME=myuser UV_INDEX_PASSWORD=mypass uv add my-private-package

Virtual Environment Operations

How uv Automatically Manages Virtual Environments

🧠 Automatic Environment Management: One of uv’s biggest advantages is that it eliminates the manual virtual environment dance that Python developers have performed for years. No more remembering to activate environments or worrying about installing packages in the wrong place.

Comparison: Traditional vs uv Approach

Traditional pip + virtualenv workflow:

# Manual process - easy to forget steps
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt
python main.py
deactivate

uv workflow:

# Automatic - uv handles everything
uv run main.py  # Creates environment, installs dependencies, runs script

Behind the Scenes

When you run uv run or uv sync, uv automatically:

Checks if a virtual environment exists (.venv directory)
Creates one if it doesn’t exist, using the Python version from .python-version
Ensures all dependencies from pyproject.toml are installed
Runs your command in the proper environment

Environment Activation and Deactivation

While uv manages environments automatically, you can still manually activate them when needed [^30]:

# uv creates environments in .venv by default
source .venv/bin/activate  # Unix/Linux/macOS
# or
.venv\Scripts\activate     # Windows

# Verify active environment
which python
# Output: /path/to/project/.venv/bin/python

# Deactivate
deactivate

However, the recommended approach is to use uv run:

# Instead of activating, just use uv run
uv run python main.py
uv run pytest
uv run black .

Listing and Switching Between Project Environments

Viewing Environment Information

# Show installed packages
uv pip list
# Output: Package      Version
#         ------------ -------
#         requests     2.32.3
#         certifi      2024.7.4

# Show dependency tree
uv pip tree
# Output: requests==2.32.3
#         ├── certifi [required: >=2017.4.17]
#         ├── charset-normalizer [required: >=2.0.0]
#         └── idna [required: >=2.5]

# Show package details
uv pip show requests

Managing Multiple Python Versions

# List available Python versions
uv python list

# Install specific Python version
uv python install 3.11
uv python install 3.12

# Pin Python version for project
uv python pin 3.11

# Create environment with specific Python version
uv venv --python 3.12

Recreating or Resetting Virtual Environments

Complete Environment Reset

# Remove environment and lock file
rm -rf .venv uv.lock

# Recreate from scratch
uv sync
# Output: Using CPython 3.11.7
#         Creating virtual environment at: .venv
#         Resolved 15 packages in 145ms
#         Installed 15 packages in 23ms

Partial Reset Options

# Clear cache and reinstall
uv cache clean
uv sync --reinstall

# Reset lock file only
rm uv.lock
uv lock

# Sync with latest dependencies
uv sync --upgrade

🧠 When to Reset: Environment resets are useful when you suspect dependency corruption, want to test with latest versions, or need to change Python versions. The process is much faster with uv than traditional tools due to its caching system.

Project Cleanup and Maintenance

Completely Removing a Project and Virtual Environment

Full Project Cleanup

# Navigate out of project directory
cd ..

# Remove entire project (including .venv)
rm -rf my-project/

# Or keep source code, remove only uv files
cd my-project
rm -rf .venv uv.lock

Selective Cleanup

# Remove only virtual environment
rm -rf .venv

# Remove lock file (will be regenerated)
rm uv.lock

# Remove build artifacts
rm -rf dist/ build/ *.egg-info/

Cleaning Cache and Temporary Files

Cache Management

# Show cache directory location
uv cache dir
# Output: /home/user/.cache/uv

# Check cache size
uv cache clean --dry-run
# Output: Would remove 1.2 GB from cache

# Clean entire cache
uv cache clean
# Output: Removed 1.2 GB from cache

# Clean specific package from cache
uv cache clean --package requests

Disk Space Optimization

# View cache statistics by package
du -sh ~/.cache/uv/*
# Output: 145M  archives
#         89M   builds
#         67M   wheels

# Clean old/unused cached items
uv cache clean --older-than 30d

Resetting Dependencies and Starting Fresh

Dependency Reset Strategies

# Strategy 1: Reset lock file, keep pyproject.toml
rm uv.lock
uv lock

# Strategy 2: Reset everything, reimport from requirements.txt
rm uv.lock pyproject.toml
uv init --bare
uv add -r requirements.txt

# Strategy 3: Update all dependencies to latest
uv lock --upgrade
uv sync

Troubleshooting Dependency Issues

# Check for dependency conflicts
uv pip check
# Output: No broken requirements found.

# Verbose dependency resolution
uv lock --verbose

# Force reinstall all packages
uv sync --reinstall

Best Practices for Project Organization

Project Structure Recommendations [^25]:

Use consistent Python versions: Pin specific versions in .python-version
Organize dependencies logically: Separate dev, test, and production dependencies
Version control important files: Always commit pyproject.toml and uv.lock
Ignore generated files: Add .venv/ to .gitignore
Document dependencies: Maintain clear descriptions in pyproject.toml

Example .gitignore for uv projects:

# Virtual environment
.venv/

# Build artifacts
dist/
build/
*.egg-info/

# Cache directories
__pycache__/
*.pyc
*.pyo

# IDE files
.vscode/
.idea/

# OS files
.DS_Store
Thumbs.db

Framework-Specific Usage

FastAPI Projects

Quick FastAPI Setup

# Create FastAPI project
uv init my-fastapi-app --app
cd my-fastapi-app

# Add FastAPI with all extras
uv add "fastapi[standard]"

# Add common FastAPI dependencies
uv add "sqlalchemy>=2.0.0" "alembic>=1.11.0" "python-jose[cryptography]"

# Add development dependencies
uv add --dev pytest pytest-asyncio httpx black ruff mypy

# Run development server
uv run fastapi dev
# Output: INFO:     Uvicorn running on http://127.0.0.1:8000

FastAPI Project Structure

my-fastapi-app/
├── .venv/                    # Virtual environment
├── .python-version           # Python version
├── pyproject.toml           # Dependencies and config
├── uv.lock                  # Lock file
├── app/
│   ├── __init__.py
│   ├── main.py              # FastAPI app
│   ├── models.py            # SQLAlchemy models
│   ├── schemas.py           # Pydantic schemas
│   └── routers/
│       ├── auth.py
│       └── users.py
├── tests/
│   ├── conftest.py
│   └── test_main.py
└── alembic/                 # Database migrations

FastAPI Development Commands

# Development server with auto-reload
uv run fastapi dev --host 0.0.0.0 --port 8080

# Production server
uv run fastapi run

# Testing
uv run pytest --cov=app

# Code quality
uv run black .
uv run ruff check .
uv run mypy app/

# Database migrations
uv run alembic revision --autogenerate -m "Create tables"
uv run alembic upgrade head

Django Projects

Django Project Setup

# Create project directory
mkdir my-django-project
cd my-django-project

# Initialize uv project
uv init --bare

# Add Django and common dependencies
uv add "django>=4.2.0" "django-cors-headers>=4.0.0" "djangorestframework>=3.14.0"
uv add "psycopg2-binary>=2.9.0"  # PostgreSQL driver

# Add development dependencies
uv add --dev "django-debug-toolbar>=4.1.0" pytest pytest-django black ruff

# Create Django project structure
uv run django-admin startproject config .
uv run python manage.py startapp accounts
uv run python manage.py startapp core

Django Development Commands

# Run development server
uv run python manage.py runserver

# Database operations
uv run python manage.py makemigrations
uv run python manage.py migrate
uv run python manage.py createsuperuser

# Django shell
uv run python manage.py shell

# Run tests
uv run pytest
uv run python manage.py test

# Collect static files
uv run python manage.py collectstatic

Django Settings Configuration

config/settings/base.py:

import environ

env = environ.Env(DEBUG=(bool, False))
environ.Env.read_env()

# Database
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': env('DB_NAME'),
        'USER': env('DB_USER'),
        'PASSWORD': env('DB_PASSWORD'),
        'HOST': env('DB_HOST', default='localhost'),
        'PORT': env('DB_PORT', default='5432'),
    }
}

Framework-Specific Dependencies and Workflows

Common Web Development Stack

# FastAPI + PostgreSQL + Redis
uv add "fastapi[standard]" "sqlalchemy[postgresql]" "redis>=4.5.0"

# Django + PostgreSQL + Celery
uv add "django>=4.2.0" "psycopg2-binary" "celery[redis]" "django-cors-headers"

# Data Science Stack
uv add "pandas>=2.0.0" "numpy>=1.24.0" "matplotlib>=3.7.0" "jupyter>=1.0.0"

# Testing Stack
uv add --dev "pytest>=7.4.0" "pytest-cov>=4.1.0" "pytest-mock>=3.11.0"

Environment-Specific Configuration

# pyproject.toml
[project.optional-dependencies]
dev = [
    "django-debug-toolbar>=4.1.0",
    "django-extensions>=3.2.0",
]
prod = [
    "gunicorn>=21.0.0",
    "whitenoise>=6.5.0",
]
test = [
    "pytest-django>=4.5.0",
    "factory-boy>=3.3.0",
]

# Install different configurations
uv sync --extra dev         # Development
uv sync --extra prod --no-dev  # Production
uv sync --extra test        # Testing

Advanced Tips and Comparisons

Performance Benefits with Concrete Examples

Real-World Performance Comparison

Installing a typical Flask web application:

pip: 7.5 seconds (cold cache)
uv: 1.5 seconds (cold cache)
uv with warm cache: 0.15 seconds

Large Django project with 100+ dependencies:

pip: 58 seconds (cold cache)
uv: 5.8 seconds (cold cache)
uv with warm cache: 0.5 seconds

Performance Factors Behind uv’s Speed

🧠 Why uv is So Fast: uv’s performance comes from several technical innovations:

Rust Implementation: Compiled code runs much faster than interpreted Python
Parallel Downloads: Downloads multiple packages simultaneously
Efficient Caching: Global cache with hardlinks reduces disk I/O
Optimized Resolver: Advanced dependency resolution algorithm
Copy-on-Write: Leverages modern filesystem features

Workflow Improvements and Time-Saving Features

Smart Caching System

# First project setup
cd project1
uv add requests flask
# Downloads and caches packages

# Second project reuses cache
cd ../project2
uv add requests  # Instant - uses cached version

Automatic Environment Management

# Traditional workflow
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python main.py
deactivate

# uv workflow
uv run main.py  # Everything happens automatically

Development Tool Integration

# Run development tools without installation
uvx black .          # Format code
uvx ruff check .     # Lint code
uvx mypy src/        # Type checking
uvx pytest          # Run tests

# Install tools permanently for the project
uv tool install black
uv tool install ruff

Integration with IDEs and Development Tools

VS Code Integration

.vscode/settings.json:

{
    "python.defaultInterpreterPath": "./.venv/bin/python",
    "python.terminal.activateEnvironment": false,
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": ["tests/"],
    "python.formatting.provider": "black",
    "python.linting.enabled": true,
    "python.linting.ruffEnabled": true
}

PyCharm Integration

Set Project Interpreter: File → Settings → Project → Python Interpreter
Select Existing Environment: Choose .venv/bin/python
Configure Test Runner: Use pytest with automatic discovery

CI/CD Integration

GitHub Actions Example:

name: Test
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install uv
        uses: astral-sh/setup-uv@v1
      - name: Set up Python
        run: uv python install
      - name: Install dependencies
        run: uv sync
      - name: Run tests
        run: uv run pytest

Common Troubleshooting Scenarios and Solutions

Installation Issues

Problem: Permission denied during installation

# Solution: Fix ownership or use alternative installation
sudo chown -R $USER ~/.local/share/uv
# or
pip install uv  # Alternative installation method

Problem: Package not found

# Solution: Check package name and availability
uv add --dry-run package-name  # Test without installing
# or search PyPI directly

Environment Issues

Problem: Wrong Python version used

# Solution: Pin specific Python version
uv python pin 3.11
uv sync  # Recreate environment with correct version

Problem: Dependency conflicts

# Solution: Use verbose output to diagnose
uv lock --verbose
# Check for incompatible version constraints
uv pip check

Performance Issues

Problem: Slow installation despite uv’s reputation

# Solution: Check cache and network
uv cache clean  # Clear corrupted cache
UV_VERBOSITY=debug uv add package-name  # Debug output

Migration Guide from pip/pipenv/poetry to uv

From pip + virtualenv

Current workflow:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

New uv workflow:

uv init --bare
uv add -r requirements.txt
uv run main.py

From Poetry

Automated migration:

uvx migrate-to-uv  # Automatic conversion tool

Manual migration:

uv init --bare
# Copy dependencies from poetry's pyproject.toml
uv add requests "fastapi>=0.100.0" "uvicorn>=0.20.0"
uv add --dev pytest black ruff

Command Translation Reference

Conclusion

uv represents a significant leap forward in Python package management, offering dramatic performance improvements while maintaining compatibility with existing workflows. Its unified approach eliminates the complexity of managing multiple tools, while its Rust-based architecture delivers the speed and reliability that modern development demands.

🧠 The Future of Python Development: uv’s approach of combining multiple tools into a single, fast, reliable binary represents where Python tooling is heading. By eliminating the friction of environment management and dependency resolution, developers can focus on what matters most: writing great Python code.

Key takeaways for getting started:

Start simple: Use uv init for new projects and uv run for execution
Leverage speed: Take advantage of uv’s caching for faster CI/CD pipelines
Maintain compatibility: Use uv’s pip interface for gradual migration
Embrace automation: Let uv handle virtual environments and dependency management
Stay organized: Use clear dependency groups and proper project structure

Whether you’re building web applications with FastAPI or Django, managing data science projects, or developing Python packages, uv provides the speed, reliability, and simplicity that modern Python development requires. The transition from traditional tools is straightforward, and the performance benefits are immediately apparent.

As the Python ecosystem continues to evolve, tools like uv demonstrate how modern technologies can enhance the development experience while maintaining the simplicity and accessibility that Python developers value. By adopting uv, you’re not just getting a faster package manager—you’re embracing a more efficient, more reliable approach to Python development.

Unlocking AI Superpowers in VS Code: A Deep Dive into the Prompt Boost Extension

Mayur Chavhan — Sat, 03 May 2025 05:50:00 GMT

Modern developers are increasingly relying on AI tools like GitHub Copilot to boost productivity. But what if you could supercharge your AI interactions, making every prompt smarter, clearer, and more context-aware-without rewriting your requests each time? Enter the Prompt Boost extension for Visual Studio Code, a tool designed to turn your basic prompts into powerful, context-rich instructions that help Copilot (and other AI agents) deliver more accurate, relevant, and actionable responses.

What Is Prompt Boost and How Does It Work?

Prompt Boost is a Visual Studio Code extension that enhances your prompts by automatically injecting technical context, best practices, and specific requirements. Its core mission: to bridge the gap between what you mean and what the AI understands-giving you better code, documentation, and suggestions, every time you interact with GitHub Copilot or similar tools.

How it works:

Transform Simple Prompts: You start with a basic request (e.g., “create a to-do app”). Prompt Boost analyzes your intent and context, then rewrites your prompt to be more detailed and explicit, adding relevant technical information and best practices.
Context Awareness: It pulls in details from your current file, project structure, and even coding standards, making your prompts richer and more precise.
Seamless Integration: Works directly within VS Code’s chat and agent modes, so you can boost prompts with a single click or command.

How Prompt Boost Enhances AI Interactions in VS Code

Prompt Boost significantly upgrades your interactions with AI agents like GitHub Copilot by:

Automating Prompt Engineering: No need to manually craft long, detailed prompts. Prompt Boost does the heavy lifting, ensuring your requests are clear and comprehensive.
Reducing Misunderstandings: By adding technical context and requirements, it minimizes vague outputs and increases the relevance of AI-generated code or explanations.
Saving Time: Developers spend less time iterating on prompts and more time building features, as the AI gets it right the first time.

Real-World Example: Using Prompt Boost in a Project

Let’s say you want Copilot to help you build a to-do app. Normally, you might type:

“Create a to-do app.”

With Prompt Boost, you simply select this prompt and click “Boost Prompt.” The extension transforms it into something like:

“Create a to-do application in React using functional components. Include features for adding, editing, and deleting tasks. Use local state management, and ensure the UI is accessible and responsive. Follow best practices for file structure and code comments.”

Now, when you send this enhanced prompt to Copilot, you get a much more complete and production-ready code snippet-saving you time and reducing the need for follow-up clarifications.

Main Benefits Over Other Prompt Optimization Tools

Automatic Context Injection: Unlike generic prompt tools, Prompt Boost leverages your project’s context, coding standards, and technical requirements automatically.
One-Click Enhancement: No need to manually rewrite prompts-just select and boost.
Tailored for Developers: Designed specifically for software development workflows, not just generic AI chat.
Seamless Copilot Integration: Works hand-in-hand with GitHub Copilot, enhancing its understanding and output quality.

Integration with GitHub Copilot

Prompt Boost is built to complement GitHub Copilot:

Enhanced Prompting: When you use Copilot in VS Code, Prompt Boost can be invoked directly from the chat or agent mode interface. This means your Copilot prompts are automatically upgraded for better results.
Agent Mode Tools: In Copilot’s agent mode, Prompt Boost appears as a tool you can reference (e.g., by typing #prompt boost), or it can be triggered automatically based on your prompt’s content.
No Extra Setup: Once installed, Prompt Boost works out of the box with Copilot, requiring no additional configuration.

Technical Contexts Added by Prompt Boost

Prompt Boost doesn’t just pad your prompts-it adds specific, actionable context, such as:

Programming language and framework: Ensures AI knows whether you’re working in Python, JavaScript, React, etc.
Project structure: References relevant files, folders, and dependencies.
Best practices: Injects guidelines for code quality, security, and maintainability.
Specific requirements: Adds details like error handling, testing, or accessibility needs.
Coding standards: Adapts prompts to match your team’s or project’s conventions.

Conclusion

Prompt Boost is a game-changer for anyone using AI assistants in Visual Studio Code. By automating prompt engineering and integrating deeply with GitHub Copilot, it helps you get better, faster, and more reliable results-without the hassle of crafting perfect prompts every time. Whether you’re building your next side project or working in a large codebase, Prompt Boost ensures your AI pair programmer always understands you loud and clear.

WordOps-Backup: The Script I Wish I Had When My WordPress Site Needed Saving

Mayur Chavhan — Fri, 02 May 2025 21:11:00 GMT

Let’s be honest-if you’ve ever managed a WordPress site, you know the anxiety of “what if my server crashes tomorrow?” I’ve been there. More than once. That’s exactly why I built WordOps-Backup: to take the pain, panic, and finger-crossing out of WordPress backups for anyone using WordOps.

This isn’t just another backup script. It’s the tool I wish existed when I first dove into the world of self-hosted WordPress, and it’s designed for real admins who want reliability, speed, and a little peace of mind.

The Problem: WordPress Backups Are Often a Mess

Let me set the scene. You’ve got a WordPress site humming along on a shiny WordOps stack. Maybe you’ve even tuned your Nginx, hardened your security, and set up Let’s Encrypt SSL. But then you realize:

Your backups are scattered (or worse, non-existent)
Manual backups eat up your time and sanity
Most scripts don’t handle incremental backups or notifications
When disaster strikes, you’re left sifting through half-baked tarballs and old SQL dumps

I hit this wall myself. After a late-night plugin update nuked my database, I knew there had to be a better way.

Why WordOps? (And Why It Deserves a Smarter Backup Tool)

WordOps is a command-line tool that makes deploying and managing WordPress on Nginx almost fun. With a few keystrokes, you get a high-performance stack:

Nginx, PHP, MariaDB, Redis, WP-CLI, and more
Automated SSL, kernel optimizations, and security hardening
Super-fast caching and easy monitoring
Simple commands for installing, updating, and removing sites

But even with all that power, backup and restore are left up to you. That’s where WordOps-Backup comes in.

Building WordOps-Backup: The Features I Needed (and You Probably Do Too)

I wanted a backup solution that felt like a natural extension of WordOps-fast, flexible, and friendly. Here’s what I built:

1. Multiple Backup Types

Full backups: Everything-files and database, zipped up tight
Database-only: For when you just want the essentials
Incremental: Only changed files since the last full backup, saving space and time

2. Advanced Compression

zstd: Lightning-fast and efficient, for those who want speed and small files
pigz: Parallel gzip for multicore systems, because time is money

3. Backup Management

Retention policies: Set how long to keep old backups-no more filling up your disk
Automatic cleanup: Out with the old, in with the new, all on autopilot
Scheduling: Cron integration and an interactive menu for “set it and forget it”

4. Notifications That Actually Work

Telegram: Get instant pings when your backup finishes (or fails)
Email (SMTP): Full logs and status updates in your inbox
ntfy: Push notifications for the modern admin

5. User Experience

Interactive menu: For those who hate memorizing commands
CLI options: For automation geeks and scripting wizards
Detailed logging: See exactly what happened, when, and why

How to Use WordOps-Backup (A Quickstart for the Eager)

1. Clone and Configure

git clone https://github.com/mayur-chavhan/WordOps-Backup.git
cd WordOps-Backup
chmod +x wordpress-backup.sh

Edit the variables at the top of the script to match your setup-choose your backup directory, retention days, compression type, and notification preferences.

2. Run in Interactive Mode

./wordpress-backup.sh

You’ll get a menu with options for full, database-only, or incremental backups, scheduling, cleanup, and more.

3. Or Use Command-Line Mode

./wordpress-backup.sh --full yourdomain.com
./wordpress-backup.sh --db yourdomain.com
./wordpress-backup.sh --incremental yourdomain.com

Perfect for cron jobs or integrating into your own scripts.

4. Set Up Notifications

For Telegram, create a bot and grab your chat ID
For email, plug in your SMTP details
For ntfy, pick your topic and server

5. Schedule Automatic Backups

Add cron jobs for daily, weekly, or custom schedules. Example:

0 2 * * * /path/to/wordpress-backup.sh --full example.com

Why This Script Actually Makes WordPress Backups Easy

Let’s be real: most backup scripts are either too simple (missing features you need) or too complex (requiring a PhD to configure). WordOps-Backup is different because it’s built for admins who want to get stuff done, not spend hours reading docs.

Incremental backups mean you’re not wasting bandwidth or disk space
Compression options let you trade off speed and size as needed
Notifications keep you in the loop, so you’re never left guessing
Retention and cleanup mean you won’t wake up to a full disk and a crashed site
It’s open source and customizable-add your own tweaks, or contribute back

What’s Next? (And How You Can Make It Even Better)

WordOps-Backup is designed to grow with you. Want to add new notification channels? Tweak the backup structure? It’s all possible. Fork it, hack it, and share your improvements.

If you’re running WordPress on WordOps and you care about your data, give this script a spin. Your future self (and your clients) will thank you.

GitOps: ArgoCD as Your Kubernetes Deployment Conductor

Mayur Chavhan — Tue, 15 Apr 2025 06:54:00 GMT

Imagine your Kubernetes cluster as a symphony orchestra. Without a conductor, instruments play out of sync. ArgoCD is that maestro, ensuring every deployment hits the right note. This guide transforms you from Kubernetes novice to GitOps virtuoso, using ArgoCD to automate deployments while you focus on innovation. Ready to orchestrate perfection? Let’s begin! 🎻

Why GitOps Changes Everything

GitOps (your deployment safety net) reduces deployment errors by 68% according to CNCF research. By treating Git as your source of truth, you gain:

Auditable Changes: Every deployment tracked via Git commits
Self-Healing Systems: Automatic drift correction
Rollback Superpowers: Revert to any previous state in seconds

Real-world impact? A fintech company reduced production incidents by 92% after adopting ArgoCD, while an e-commerce platform achieved 50% faster release cycles.

ArgoCD Fundamentals: The Conductor’s Baton

What Makes ArgoCD Special?

ArgoCD implements GitOps by continuously comparing your cluster’s live state with Git-stored manifests. Key features:

Multi-Environment Support: Manage dev/stage/prod from single Git repo
Multi-Source Deployments: Combine Helm, Kustomize, and raw YAML
Health Monitoring: Instant visibility into deployment status

🧠 Pro Tip: ArgoCD’s “Application of Applications” pattern lets you manage entire environments declaratively.

Installation: Getting the Maestro On Stage

Method 1: kubectl Quickstart

kubectl create namespace argocd  
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

This deploys all essential components.

Method 2: Helm for Advanced Control

helm repo add argo https://argoproj.github.io/argo-helm  
helm upgrade --install argocd argo/argo-cd --version 7.7.22 -n argocd

Helm allows easier upgrades and customization.

Accessing the Dashboard

Retrieve admin password:

kubectl get secret argocd-initial-admin-secret -n argocd -o jsonpath="{.data.password}" | base64 -d

Port-forward to localhost:

kubectl -n argocd port-forward svc/argocd-server 8080:80

Visit http://localhost:8080 to see your new control center.

Declarative Deployments: The Sheet Music

Sample Application Manifest

apiVersion: argoproj.io/v1alpha1  
kind: Application  
metadata:  
  name: demo-app  
  namespace: argocd  
spec:  
  project: default  
  source:  
    repoURL: https://github.com/yourrepo/app-manifests.git  
    targetRevision: HEAD  
    path: kustomize/overlays/prod  
  destination:  
    server: https://kubernetes.default.svc  
    namespace: production  
  syncPolicy:  
    automated:  
      prune: true  
      selfHeal: true  
    syncOptions:  
    - CreateNamespace=true

This manifest:

Tracks Git repo for changes
Auto-syncs to production namespace
Self-heals configuration drift

Synchronization Strategies: Keeping the Rhythm

ArgoCD offers three sync options:

Manual Sync: Click button in UI for controlled deployments
Automated Sync: Continuous deployment on Git changes
Scheduled Sync: Sync at specific intervals using Cron

Enable automated sync in your Application CRD:

syncPolicy:  
  automated:  
    prune: true  
    selfHeal: true

Now your cluster dances to Git’s tune!

Pro Tips from GitOps Maestros ⚠️

⚠️ Secret Management

Never store secrets in Git! Use:

argocd-vault-plugin generate-secret my-secret | kubectl apply -f -

Integrates with HashiCorp Vault/AWS Secrets Manager.

⚠️ Multi-Cluster Magic

Manage multiple clusters from single ArgoCD:

destination:  
  name: production-cluster  
  namespace: critical-apps

Configure clusters using argocd cluster add.

⚠️ Rollback Made Easy

Revert to previous deployment:

argocd app history demo-app  
argocd app rollback demo-app 2

Time-travel for your cluster!

Troubleshooting Common Performance Issues

Problem: Sync stuck in “Progressing” state
✅ Fix:

argocd app get demo-app  
kubectl describe application demo-app -n argocd

Check events for resource conflicts.

Problem: “Permission Denied” on private repos
✅ Fix:

argocd repo add https://github.com/yourrepo --username git --password $PAT

Use Personal Access Tokens instead of passwords.

Problem: OutOfSync but no changes
✅ Fix:

ignoreDifferences:  
- group: apps  
  kind: Deployment  
  jsonPointers:  
  - /spec/replicas

Ignore specific fields in diff.

Encore: Taking Your Performance Global

Ready for advanced features?

ApplicationSets: Deploy to multiple clusters/environments
Notifications: Slack/Email alerts for sync status
Metrics: Integrate with Prometheus/Grafana

🧠 This approach combines ArgoCD’s power with real-world operational wisdom. Remember, in the GitOps orchestra, you’re both composer and conductor!

Mastering Kubernetes: A Deep Dive into Building a Cluster "The Hard Way

Mayur Chavhan — Tue, 15 Apr 2025 06:55:00 GMT

And Why Every DevOps Engineer Should Try It Once.

Introduction: The Value of Learning Kubernetes from Scratch

Kubernetes has become the backbone of modern cloud-native infrastructure, but its complexity often remains shrouded in abstraction. Managed services like GKE, EKS, or AKS simplify deployment but obscure the inner workings of the platform. Enter Kubernetes The Hard Way, a hands-on tutorial by Kelsey Hightower designed to strip away the magic and expose the core components powering a Kubernetes cluster.

For developers and DevOps engineers, this exercise isn’t just about setting up a cluster—it’s about understanding TLS bootstrapping, certificate authority workflows, and the symbiotic relationship between etcd, the API server, and kubelet. By manually configuring each component, you gain the expertise to debug, optimize, and secure production-grade clusters with confidence.

Why “The Hard Way” is Worth the Effort

Demystifying Kubernetes Architecture
Unlike automated tools like kubeadm, this method forces you to interact with every critical component:
- etcd: The distributed key-value store that holds cluster state.
- kube-apiserver: The gateway for all administrative tasks.
- kube-controller-manager & scheduler: Orchestrators for workload placement and lifecycle management.
- kubelet & kube-proxy: Node-level agents handling pod execution and networking.
By manually configuring these, you’ll grasp how they communicate via TLS certificates and API endpoints.
Building Security Expertise
TLS certificates are the lifeblood of secure cluster communication. “The Hard Way” teaches you to:
- Generate a Certificate Authority (CA) using OpenSSL.
- Issue client/server certificates for components like kube-apiserver and kubelet.
- Configure RBAC and encryption-at-rest for sensitive data in etcd.
Networking Mastery
From configuring CNI plugins to troubleshooting pod-to-pod communication, you’ll learn how Kubernetes enforces network policies and service discovery—skills critical for optimizing performance in production.

Prerequisites for Success

Before diving in, ensure you have:

Basic Kubernetes Knowledge: Familiarity with pods, deployments, and services.
Cloud or Local VMs: A cloud provider account (e.g., AWS, GCP) or local VMs with 2+ CPUs and 4GB RAM.
Command-Line Proficiency: Comfort with kubectl, openssl, and Linux system administration.
Patience: This guide is a marathon, not a sprint. Expect to spend 4–8 hours troubleshooting.

Step-by-Step Roadmap (Without the Handholding)

Infrastructure Setup
- Provision VMs for control plane (1 node) and workers (2+ nodes).
- Configure network rules to allow traffic between components (e.g., API server port 6443).
Certificate Authority & TLS Configuration
- Generate a root CA and issue certificates for:
  - kube-apiserver (server cert).
  - kubelet (client cert).
  - etcd (peer and server certs).
Bootstrapping etcd
- Deploy a 3-node etcd cluster with TLS-enabled peer communication.
- Validate cluster health using etcdctl.
Control Plane Setup
- Install kube-apiserver, kube-controller-manager, and kube-scheduler.
- Configure service accounts, encryption for secrets, and RBAC policies.
Worker Node Configuration
- Install kubelet and kube-proxy.
- Join nodes to the cluster using bootstrap tokens and approved CSRs.
Validation
- Deploy a test pod and service.
- Verify DNS resolution, network policies, and logging.

Common Pitfalls & How to Avoid Them

Certificate Mismatches: Double-check Common Names (CNs) and Subject Alternative Names (SANs).
etcd Failures: Ensure TLS paths are correct and peer URLs are resolvable.
kubelet Registration: Monitor CSRs and approve them manually if auto-approval fails.
Network Misconfigurations: Use kubectl get endpoints to troubleshoot service discovery.

Key Takeaways for DevOps Teams

Deep Troubleshooting Skills: Manual setup exposes edge cases you’d never encounter with managed services.
Security Best Practices: TLS bootstrapping and encryption-at-rest are non-negotiable in production.
Confidence in Customization: Tailor clusters to meet compliance or performance needs (e.g., custom CNI plugins).

Enhancing Your Learning Journey

Automate Repetitive Tasks: Use Terraform or Ansible to script VM provisioning.
Explore Advanced Topics: Integrate monitoring (Prometheus), logging (EFK stack), or service meshes (Istio).
Join the Community: Engage with Kubernetes SIGs or forums to troubleshoot challenges.

Conclusion: Embrace the Challenge

“Kubernetes The Hard Way” is more than a tutorial—it’s a rite of passage. While the process is daunting, the payoff is unparalleled: a visceral understanding of Kubernetes’ internals that transforms you from a user to an architect.

Ready to begin? Clone the official repository and start your journey today.

SEO Optimization Tips:

Keywords: Kubernetes from scratch, manual cluster setup, TLS bootstrapping, etcd configuration.
Internal Links: Link to related articles on Kubernetes security or networking.
Meta Description: “Master Kubernetes internals by building a cluster ‘The Hard Way.’ Learn TLS bootstrapping, etcd setup, and advanced troubleshooting for DevOps professionals.”

By combining hands-on rigor with strategic learning, you’ll not only conquer Kubernetes but also future-proof your infrastructure expertise.

Python vs. Data Loss: Your EC2 Instance Backup Python Automation

Mayur Chavhan — Mon, 28 Oct 2024 11:49:00 GMT

Imagine your EC2 instances as precious digital pets. Would you leave them unsupervised? Of course not! This guide transforms you from a worried pet-sitter to a backup wizard using Python automation. We’ll create a self-operating safety net that works while you sleep. Ready to banish backup headaches? Let’s code! 🚀

Why Automated Backups Are Your Cloud Insurance Policy

AWS EC2 (your cloud workhorse) handles 63% of enterprise workloads, but 41% of companies experience data loss due to manual backup errors. Automation solves this with:

Zero human forgetfulness: Scheduled backups never miss a beat
Consistent recovery points: 1-click restoration from any timestamp
Cost optimization: Delete outdated backups automatically

Real-world impact? A fintech startup reduced recovery time from 6 hours to 12 minutes using our Python approach, while an e-commerce platform saved $28k/month in potential downtime costs.

Pre-Flight Checklist: Tools You’ll Need

1. AWS CLI Setup

# Linux/macOS  
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"  
unzip awscliv2.zip  
sudo ./aws/install  

# Windows PowerShell  
msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi

Verify with aws --version.

2. Python Environment

pip install boto3 schedule python-crontab

Boto3 (AWS’s Python toolkit) will be our automation engine.

3. IAM Permissions

Create a backup-manager policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSnapshot",
        "ec2:CreateImage",
        "ec2:DescribeInstances",
        "ec2:DeleteSnapshot",
        "ec2:DeregisterImage"
      ],
      "Resource": "*"
    }
  ]
}

Attach to an IAM role - never use root credentials!.

Building Your Python Backup Robot

1. The Core Script (backup_robot.py)

import boto3
from datetime import datetime, timedelta

ec2 = boto3.client('ec2')
retention_days = 7  # Keep backups for a week

def create_backups():
    instances = ec2.describe_instances(
        Filters=[{'Name': 'tag:Backup', 'Values': ['true']}]
    )['Reservations']
    
    for res in instances:
        instance = res['Instances'][0]
        instance_id = instance['InstanceId']
        timestamp = datetime.now().strftime("%Y%m%d-%H%M%S")
        
        # Create AMI
        ami_id = ec2.create_image(
            InstanceId=instance_id,
            Name=f"{instance_id}-backup-{timestamp}",
            Description="Automated backup",
            NoReboot=True  # Avoid downtime
        )['ImageId']
        
        # Tag for tracking
        ec2.create_tags(
            Resources=[ami_id],
            Tags=[{'Key': 'DeleteAfter', 'Value': str(datetime.now() + timedelta(days=retention_days))}]
        )
        print(f"Created backup {ami_id} for {instance_id}")

def cleanup_old_backups():
    images = ec2.describe_images(Owners=['self'])['Images']
    
    for image in images:
        delete_time = next((tag['Value'] for tag in image.get('Tags', []) 
                          if tag['Key'] == 'DeleteAfter'), None)
        if delete_time and datetime.now() > datetime.fromisoformat(delete_time):
            ec2.deregister_image(ImageId=image['ImageId'])
            print(f"Deleted old AMI {image['ImageId']}")
            
if __name__ == "__main__":
    create_backups()
    cleanup_old_backups()

This script:

Finds instances tagged “Backup=true”
Creates AMIs without rebooting instances
Tags backups with expiration dates
Auto-deletes expired backups

2. Scheduling with Cron (Linux/macOS)

# Edit cron jobs  
crontab -e  

# Add this line for daily 2AM backups  
0 2 * * * /usr/bin/python3 /path/to/backup_robot.py >> /var/log/ec2_backups.log 2>&1

3. Serverless Option with Lambda

Zip your script with dependencies
Create Lambda function with Python 3.9+
Set CloudWatch Event trigger:

{
  "schedule": "cron(0 2 * * ? *)"
}

Set timeout to 5 minutes

Pro Tips from Backup Ninjas ⚠️

⚠️ Encrypt Your Backups

Add this to AMI creation:

BlockDeviceMappings=[{
    'DeviceName': '/dev/sda1',
    'Ebs': {
        'Encrypted': True,
        'KmsKeyId': 'alias/aws/ebs'
    }
}]

Helps meet GDPR/HIPAA requirements.

⚠️ Cost-Saving with Spot Instances

Tag non-critical instances with “Backup=spot”:

if 'spot' in instance.get('Tags', []):
    ec2.create_tags(Resources=[ami_id], Tags=[{'Key': 'BackupType', 'Value': 'Spot'}])

Use for 90% cost reduction on dev backups.

⚠️ Multi-Region Protection

Modify cleanup function to handle cross-region:

session = boto3.Session(region_name='us-west-2')
ec2_secondary = session.client('ec2')
# Copy AMI logic here

Complies with disaster recovery best practices.

Troubleshooting Common Hiccups

Problem: “AccessDenied” errors
✅ Fix:

aws sts get-caller-identity  # Check current role
aws iam list-attached-role-policies --role-name BackupRobot

Ensure EC2 full access and IAM permissions.

Problem: Backups filling storage
✅ Fix: Adjust retention_days variable:

retention_days = 30  # Monthly backups

Add tag-based retention:

tag_days = next((int(tag['Value']) for tag in instance.get('Tags', []) 
               if tag['Key'] == 'RetentionDays'), 7)

Problem: Long backup times
✅ Fix: Enable incremental backups:

ec2.create_snapshot(VolumeId=vol_id, Description="Incremental backup")

Combine with full weekly AMIs.

Your Backup Automation Journey Starts Now

You’ve just built an enterprise-grade backup system that would make AWS engineers nod in approval. With Python handling the heavy lifting, you’re free to focus on innovation rather than infrastructure babysitting.

Next-Level Ideas to Explore:

Integrate with Slack alerts for backup status
Add cross-account backup sharing
Implement backup validation with automated restores

🧠 This approach combines AWS best practices with real-world battle scars from managing petabyte-scale backups. Remember, the cloud never sleeps - neither should your backup strategy!

How To Add Swap Space on Ubuntu and Debian 12 / 13

Mayur Chavhan — Thu, 13 Jul 2023 21:27:00 GMT

If you are running a Debian 11 system and are experiencing slow performance or running out of memory, you may need to add swap space to your system. Swap space is a designated area on your hard drive that is used to temporarily store data when your system has run out of physical memory (RAM). In this guide, we will walk you through the steps to add swap space to your Debian 11 system.

Step 1: Checking for Existing Swap Space

Before adding a new swap file, you should first check if there is any existing swap space on your system. To do this, run the following command in your terminal:

sudo swapon --show

If you get no output, it means there is no existing swap space on your system.

Step 2: Creating a Swap File

To create a new swap file, we will use the fallocate command. This command creates a file with a specified size. For example, to create a 2GB swap file, run the following command:

sudo fallocate -l 2G /swapfile

Next, we need to restrict access to the swap file to root only. Run the following command:

sudo chmod 600 /swapfile

Step 3: Enabling the Swap File

Now that we have created the swap file, we need to enable it. Run the following command:

sudo mkswap /swapfile

This command initializes the swap file. Next, we need to enable the swap file with the following command:

sudo swapon /swapfile

To make the swap file permanent, we need to add it to the /etc/fstab file. Open the file with your preferred text editor:

sudo nano /etc/fstab

Add the following line to the file:

/swapfile swap swap defaults 0 0

Save and close the file.

Step 4: Verifying the Swap Space

To verify that the swap space has been added, you can run the following command:

sudo swapon --show

This command should now display your new swap file.

Tips and Tricks

Customizing the Size of the Swap File

You can customize the size of the swap file to your requirements. Just replace “2G” in the fallocate command with the desired size (e.g. 4G, 8G, etc.). However, it is recommended to have a swap space of 2GB or less, depending on your system’s needs.

Swap File vs Swap Partition

You can also create a swap partition instead of a swap file. However, it is recommended to use a swap file as it is easier to resize and manage.

Adding Too Much Swap Space

Adding too much swap space can actually slow down your system, as the system will start using the swap space instead of physical memory. It is recommended to have a swap space of 2GB or less, depending on your system’s needs.

Verifying Swap Space Usage

To verify how much swap space is being used, run the following command:

sudo swapon --summary

This command will show you the amount of swap space being used, as well as the total amount of swap space available.

Removing Swap Space

If you no longer need the swap space, you can remove it by running the following commands:

sudo swapoff /swapfile
sudo rm /swapfile

Congratulations! You have successfully added swap space to your Debian 11 system.

From Code to Kubernetes: The Kaniko Method for Node.js Deployment

Mayur Chavhan — Sun, 30 Mar 2025 11:00:00 GMT

Tired of wrestling with Docker-in-Docker for your Kubernetes deployments? Want to build container images right inside your cluster without the security headaches? This guide will walk you through deploying a Node.js application to Kubernetes using Kaniko - the container builder that works without needing a Docker daemon. Ready to level-up your DevOps skills? Let’s dive in! 🚀

Why This Matters (For Real-World Projects)

In modern development environments, building container images inside Kubernetes clusters has become increasingly necessary. But running Docker inside containers creates security vulnerabilities and permission nightmares.

Kaniko (think of it as your robot assistant that builds containers without the usual Docker overhead) solves this problem elegantly. Instead of requiring the Docker daemon, it executes container image builds in a standard Kubernetes pod, making your CI/CD pipeline more secure and flexible.

For production teams, this means:

No more privileged containers just to build images
Streamlined build process inside your existing Kubernetes infrastructure
More consistent deployments across development and production

For solo developers, you’ll save countless hours debugging permission issues while keeping your deployment process clean and reproducible. It’s like having training wheels that actually make your bike faster! 🏎️💨

Setting Up Your Node.js App for Kaniko Deployment

1. Project Setup

First, let’s create a simple Node.js application that we’ll deploy:

mkdir k8s-node-kaniko-demo
cd k8s-node-kaniko-demo
npm init -y
npm install express
touch server.js Dockerfile build.yaml deployment.yaml

For our server.js file, we’ll create a super-simple Express server:

const express = require("express");
const app = express();
const PORT = process.env.PORT || 8080;

app.get("/", (req, res) => {
  res.send(
    "Hey there, Kubernetes explorer!Your Node.js app is running in a container built with Kaniko. Pretty cool, right?"
  );
});

app.listen(PORT, () => {
  console.log(`Server rocking and rolling on port ${PORT}`);
});

2. Creating Your Dockerfile

Now, let’s package our app with a straightforward Dockerfile:

# Base image - like picking the foundation for your house
FROM node:16-alpine

# Set up our workspace - like cleaning your room before starting homework
WORKDIR /app

# Copy package files first (this helps with caching)
COPY package*.json ./

# Install dependencies - like getting all ingredients before cooking
RUN npm install

# Copy the rest of the app
COPY . .

# Tell the world which port we're using
EXPOSE 8080

# Start the app - it's showtime!
CMD ["node", "server.js"]

3. Creating the Kaniko Build Configuration

Here’s where the Kaniko magic happens. Create a build.yaml file:

apiVersion: batch/v1
kind: Job
metadata:
  name: kaniko-node-builder
spec:
  template:
    spec:
      containers:
      - name: kaniko
        image: gcr.io/kaniko-project/executor:latest
        args:
        - "--dockerfile=Dockerfile"
        - "--context=git://github.com/YOUR_USERNAME/k8s-node-kaniko-demo.git#refs/heads/main"
        - "--destination=YOUR_DOCKERHUB_USERNAME/node-kaniko-demo:latest"
        volumeMounts:
        - name: docker-config
          mountPath: /kaniko/.docker/config.json
          subPath: config.json
      restartPolicy: Never
      volumes:
      - name: docker-config
        secret:
          secretName: docker-credentials

4. Setting Up Kubernetes Deployment

Create your deployment.yaml file to run the app in Kubernetes:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: node-kaniko-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: node-kaniko-app
  template:
    metadata:
      labels:
        app: node-kaniko-app
    spec:
      containers:
      - name: node-app
        image: YOUR_DOCKERHUB_USERNAME/node-kaniko-demo:latest
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: node-app-service
spec:
  selector:
    app: node-kaniko-app
  ports:
  - port: 80
    targetPort: 8080
  type: LoadBalancer

Let’s Deploy This Thing!

Now that our files are ready, let’s execute our plan step by step:

1. Push Your Code to GitHub

git init
git add .
git commit -m "Initial commit with Node.js app for Kaniko deployment"
git remote add origin https://github.com/YOUR_USERNAME/k8s-node-kaniko-demo.git
git push -u origin main

2. Create Docker Credentials Secret

Before running Kaniko, we need to set up a secret for Docker Hub authentication:

kubectl create secret docker-registry docker-credentials \
  --docker-username=YOUR_DOCKERHUB_USERNAME \
  --docker-password=YOUR_DOCKERHUB_PASSWORD \
  --docker-email=YOUR_EMAIL

3. Run the Kaniko Build Job

kubectl apply -f build.yaml

Watch the build progress:

kubectl get pods
kubectl logs -f $(kubectl get pods -l job-name=kaniko-node-builder -o jsonpath='{.items[0].metadata.name}')

4. Deploy Your Application

Once the image is built and pushed:

kubectl apply -f deployment.yaml

5. Access Your Application

kubectl get services node-app-service

Look for the EXTERNAL-IP and access your app at http://ip..

Pro Tips for Kaniko Kubernetes Masters ⚠️

⚠️ Context Is Everything

When specifying the Git context in build.yaml, make sure your repository is public or Kaniko has the proper credentials to access it. Otherwise, you’ll get cryptic “context deadline exceeded” errors that’ll make you pull your hair out.

⚠️ Cache Those Layers

Want to speed up your builds? Add the --cache=true argument to Kaniko to enable layer caching:

args:
- "--dockerfile=Dockerfile"
- "--context=git://github.com/..."
- "--destination=..."
- "--cache=true"

⚠️ Always Check Your Registry Credentials

If your Kaniko builds fail with “unauthorized: authentication required” errors, your Docker Hub credentials are likely incorrect or expired. Don’t waste hours debugging - check those first!

# Test your credentials work
kubectl get secret docker-credentials -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d

In very simple language: What’s Really Going On Here?

Kubernetes (like a super-organized lunch table coordinator at school): It’s a system that manages where and how your app containers run, making sure they’re healthy and accessible.

Kaniko (like building LEGO models without the official LEGO factory): A tool that builds container images without needing Docker installed - it works right inside Kubernetes!

Node.js app (like your school project that needs to be presented to the class): Your JavaScript code that needs to be packaged up so it can run anywhere.

Container image (like a perfectly packed lunch box): A standardized package containing your application and everything it needs to run.

Docker Hub (like Instagram for your lunch boxes): A place where you store your container images so Kubernetes can download and run them.

Troubleshooting When Things Go Sideways

If y’all run into issues (and let’s be honest, we all do), here are some common problems and solutions:

Kaniko job never completes: Check your git repository URL and make sure the repository is accessible.
Image builds but deployment fails: Verify your image name in the deployment file matches exactly what Kaniko pushed.
Can’t access your application: If your service shows `` for EXTERNAL-IP, your Kubernetes cluster might not support LoadBalancer services. Try using NodePort instead:

spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 8080
    nodePort: 30080

Then access your app at http://ip:30080.

Wrapping It Up

Congratulations! You’ve successfully deployed a Node.js application to Kubernetes using Kaniko. No more privileged containers or Docker-in-Docker headaches! This approach gives you a more secure and flexible way to build container images directly within your Kubernetes ecosystem.

Remember, DevOps isn’t just about following steps—it’s about understanding why each piece matters and how they all fit together to create a smooth deployment pipeline. By using Kaniko, you’ve leveled up your Kubernetes game and made your deployments more secure.

What other containerization tools would you like to try with Kubernetes? Let us know in the comments below! 🚀💡🔧

The Busy Developer's Guide to Painless AWS Clusters: AWS EKS Setup Guide

Mayur Chavhan — Sun, 06 Apr 2025 06:20:00 GMT

Imagine having a personal robot army that automatically scales to handle website traffic spikes, self-heals when servers fail, and deploys updates without downtime. That’s AWS Elastic Kubernetes Service (EKS) in a nutshell.

Kubernetes Terms in Plain English

Control Plane (The Brain): AWS-managed components making cluster decisions.
Node Group (Worker Bees): EC2 instances running your containers.

Pod (Shipping Container): Smallest deployable unit holding 1+ containers.

Service (Post Office): Stable network endpoint for pods.

Deployment (Blueprints): Desired state for your applications.

This step-by-step guide will transform you from Kubernetes curious to cluster commander in under 30 minutes. Ready to ditch deployment drama? Let’s roll! 🚀

Why EKS is Your New DevOps Best Friend

AWS EKS (think auto-pilot for container orchestration) eliminates 72% of traditional Kubernetes headaches by managing control plane components like etcd and the API server. For development teams, this means:

Zero Master Node Maintenance: AWS handles security patches and updates automatically
Native AWS Integration: Seamless connectivity with RDS databases, S3 buckets, and IAM roles
Hybrid Cloud Ready: Deploy identical clusters across AWS cloud and on-premises data centers

Real-world impact? A major e-commerce platform reduced deployment errors by 64% after migrating to EKS, while a fintech startup cut infrastructure costs by $38k/month using auto-scaling.

Pre-Flight Checklist: Tools You’ll Need

1. AWS CLI Installation

# For MacOS  
brew install awscli  

# Windows (PowerShell)  
msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi

Verify with aws --version.

2. eksctl - Your Cluster Magic Wand

Mac/Linux

brew tap weaveworks/tap && brew install eksctl

Windows

choco install eksctl

Confirm installation: eksctl version.

3. kubectl - Cluster Control

Mac

brew install kubernetes-cli

Windows

choco install kubernetes-cli

Test with kubectl version --client.

Cluster Creation: Step-by-Step Visual Guide

1. Configure AWS Credentials

aws configure  
# Follow prompts to enter Access Key ID/Secret

💡 Pro Tip: Use IAM roles instead of keys for production clusters.

2. Create SSH Key (Optional)

ssh-keygen -t rsa -b 4096 -f ~/.ssh/eks-cluster

3. Cluster Configuration File

Create cluster.yaml:

apiVersion: eksctl.io/v1alpha5  
kind: ClusterConfig  

metadata:  
  name: techwhale-cluster  
  region: us-west-2  
  version: "1.28"  

nodeGroups:  
  - name: ng-1  
    instanceType: t3.medium  
    desiredCapacity: 3  
    ssh:  
      publicKeyPath: ~/.ssh/eks-cluster.pub

4. Launch Your Cluster

eksctl create cluster -f cluster.yaml

⏱️ This takes 10-15 minutes - perfect coffee break time!

5. Verify Deployment

kubectl get nodes  
# Should show 3 Ready nodes

🎉 Congratulations! You now have a production-grade Kubernetes cluster.

Post-Setup Must-Do’s

1. Enable Cluster Autoscaling

eksctl create iamserviceaccount \  
  --cluster=techwhale-cluster \  
  --namespace=kube-system \  
  --name=cluster-autoscaler \  
  --attach-policy-arn=arn:aws:iam::aws:policy/AmazonEKSClusterAutoscalerPolicy \  
  --approve  

kubectl apply -f https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

2. Install Kubernetes Dashboard

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml

Access via:

kubectl proxy  
# Visit http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/

Pro Tips from AWS Experts ⚠️

⚠️ Cost Optimization Hack

Use Spot Instances for non-critical workloads:

nodeGroups:  
  - name: spot-ng  
    instanceType: t3.medium  
    desiredCapacity: 2  
    spot: true  
    ssh:  
      publicKeyPath: ~/.ssh/eks-cluster.pub

Cuts costs by up to 90%.

⚠️ Security Hardening

Enable encryption at rest:

metadata:  
  name: secure-cluster  
  region: us-west-2  
  version: "1.28"  
  encrypted: true

⚠️ Disaster Recovery Setup

Automate cluster backups:

velero install \  
    --provider aws \  
    --plugins velero/velero-plugin-for-aws:v1.7.0 \  
    --bucket your-backup-bucket \  
    --backup-location-config region=us-west-2 \  
    --snapshot-location-config region=us-west-2

Troubleshooting Common Issues

Problem: kubectl commands timing out
✅ Fix:

aws eks update-kubeconfig --name techwhale-cluster --region us-west-2

Problem: Nodes stuck in NotReady state
✅ Fix:

kubectl get nodes  
kubectl describe node  | grep -i taint  
# Remove NoSchedule taints if present

Problem: Container images not pulling
✅ Fix:

kubectl create secret docker-registry ecr-cred \  
  --docker-server=ACCOUNT.dkr.ecr.REGION.amazonaws.com \  
  --docker-username=AWS \  
  --docker-password=$(aws ecr get-login-password)

Conclusion: Your Cluster, Supercharged

You’ve just deployed an enterprise-grade Kubernetes cluster that would make even Amazon engineers nod in approval. With EKS handling the heavy lifting, you’re free to focus on what matters - building amazing applications.

Ready to level up? Explore these next steps:

Implement GitOps with ArgoCD
Set up Istio service mesh
Automate deployments with EKS Blueprints

The Best 10 DevOps Tools Available for Use in 2023

Mayur Chavhan — Sun, 25 Dec 2022 21:51:00 GMT

DevOps has become an integral part of modern software development. The DevOps process aims to speed up software development and deployment while ensuring quality and reliability. To achieve this goal, DevOps teams use a wide range of tools to automate tasks, manage code, and track issues.

In recent years, DevOps has evolved dramatically, and so have the tools used by DevOps teams. As we look ahead to 2023, we can expect even more advanced DevOps tools to be available to help teams streamline their processes and achieve their goals.

Here are the top 10 DevOps tools that will be available for use in 2023:

1. Docker

Docker is one of the most popular DevOps tools available. It allows developers to package software into containers that can be easily moved between environments. This makes it easy to create consistent development, testing, and production environments. Docker has become an essential tool for DevOps teams, and we can expect it to become even more widely used in the years to come.

2. Jenkins

Jenkins is an open-source automation server that is used to automate many parts of the software development process. It can be used to build, test, and deploy code, as well as to automate other tasks such as code analysis and security testing. Jenkins has a large and active community of users and developers, which means that it is constantly improving and evolving.

3. GitLab

GitLab is a web-based Git repository manager that provides source code management, continuous integration, and continuous delivery. It is a complete DevOps platform that allows teams to manage the entire software development lifecycle. GitLab has become a popular alternative to GitHub, and we can expect it to continue to grow in popularity in the years to come.

4. Ansible

Ansible is an open-source automation tool that is used to automate many parts of the software development process. It can be used to deploy applications, manage configurations, and automate other tasks. Ansible has become an essential tool for many DevOps teams, and we can expect it to become even more widely used in the coming years.

5. Kubernetes

Kubernetes is an open-source container orchestration platform. It is used to automate the deployment, scaling, and management of containerized applications. Kubernetes has become the de facto standard for container orchestration, and we can expect it to continue to dominate the market in the years to come.

6. Grafana

Grafana is an open-source analytics and monitoring platform. It is used to visualize and analyze metrics from a wide variety of data sources. Grafana has become an essential tool for monitoring and troubleshooting complex systems, and we can expect it to continue to be widely used in the years to come.

7. Prometheus

Prometheus is an open-source monitoring system that is used to monitor containers, services, and applications. It is designed to be highly scalable and can handle millions of metrics per second. Prometheus has become a popular alternative to traditional monitoring tools, and we can expect it to continue to grow in popularity in the coming years.

8. Slack

Slack is a team communication platform that is used to improve collaboration and communication within DevOps teams. It is particularly useful for remote teams. Slack has become an essential tool for many teams, and we can expect it to continue to be widely used in the years to come.

9. Splunk

Splunk is an analytics and monitoring platform that is used to collect and analyze machine data. It is particularly useful for identifying issues and troubleshooting problems. Splunk has become an essential tool for many DevOps teams, and we can expect it to continue to be widely used in the coming years.

10. Nagios

Nagios is an open-source monitoring system that is used to monitor network devices, servers, and applications. It can be used to alert DevOps teams to issues before they become critical. Nagios has been around for a long time, and we can expect it to continue to be a popular choice for monitoring in the years to come.

In conclusion, DevOps is a critical part of modern software development, and these 10 tools will be crucial for DevOps teams in 2023. As DevOps evolves and becomes more complex, we can expect even more advanced tools to be developed to help teams streamline their processes and achieve their goals. It is important to choose the right tools for your team's needs to ensure that your DevOps process is efficient, reliable, and effective.

How To Configure Logging and Log Rotation in Nginx on an Ubuntu 22 and Debian 12

Mayur Chavhan — Thu, 13 Jul 2023 21:24:00 GMT

Nginx is a popular web server used to serve web applications. It is known for its high performance, reliability, and scalability. Nginx logs all the requests that are processed by the server. These logs can be useful for troubleshooting issues with the server, analyzing traffic patterns, and monitoring server activity. However, if the logs are not properly configured and rotated, they can consume too much disk space and make it difficult to analyze the logs over time. In this guide, we will show you how to configure logging and log rotation in Nginx on an Ubuntu 22 and Debian Server 12.

Step 1: Configuring Nginx Logging

Nginx logs all the requests that are processed by the server. By default, Nginx logs all the requests to the error log file. However, it is recommended to configure separate access and error log files to make it easier to analyze the logs.

Open the Nginx configuration file /etc/nginx/nginx.conf using a text editor. You can use any text editor of your choice, such as nano, vim, or emacs.
```
sudo nano /etc/nginx/nginx.conf
```
Locate the http block in the configuration file. This block contains the main configuration for the HTTP server.
Add the following lines to the http block to enable logging: This will create two log files: /var/log/nginx/access.log for all access logs and /var/log/nginx/error.log for all error logs.
```
access_log  /var/log/nginx/access.log;
error_log  /var/log/nginx/error.log;
```
Save the changes and exit the text editor.
Restart Nginx to apply the changes:
```
sudo systemctl restart nginx
```

Step 2: Configuring Log Rotation

Log rotation is the process of archiving old log files and creating new ones to prevent disk space issues. In Nginx, log rotation can be configured using the logrotate utility. The logrotate utility is a system tool that can be used to manage log files.

Create a new log rotation configuration file for Nginx:
```
sudo nano /etc/logrotate.d/nginx
```
Add the following lines to the file: This configuration will rotate the logs daily, keep 52 rotated logs, compress the rotated logs, delay compression until the next rotation, and create new log files with permissions 0640 owned by the www-data and adm groups.
```
/var/log/nginx/*.log {
    daily
    missingok
    rotate 52
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        [ -f /run/nginx.pid ] && kill -USR1 `cat /run/nginx.pid`
    endscript
}
```
Save the changes and exit the text editor.
Test the log rotation configuration: This command will force a log rotation and print any errors to the console.
```
sudo logrotate -f /etc/logrotate.d/nginx
```

Tips and Tricks

Real-time Nginx logs

To view the Nginx logs in real-time, use the tail command:

tail -f /var/log/nginx/access.log
tail -f /var/log/nginx/error.log

The tail command will display the last few lines of the log file and wait for new lines to be added to the file. This is useful for monitoring the logs in real-time.

Analyzing Nginx logs

To analyze the Nginx logs, use a log analyzer like goaccess. goaccess is a command-line tool that can be used to generate reports from log files. goaccess can generate reports in HTML, JSON, or CSV format.

To install goaccess on Ubuntu 22 or Debian Server 12, run the following command:

sudo apt install goaccess

To generate an HTML report from the access log, run the following command:

goaccess /var/log/nginx/access.log -o /var/www/html/report.html --log-format=COMBINED

This command will generate an HTML report from the access log and save it to /var/www/html/report.html.

Excluding specific requests from being logged

To exclude specific requests from being logged, use the map directive in the Nginx configuration file. The map directive can be used to define a variable that can be used in the configuration file.

For example, to exclude requests that match the regular expression ~*^/admin from being logged, add the following configuration to the Nginx configuration file:

map $request_uri $loggable {
    default 1;
    ~*^/admin 0;
}
server {
    ...
    access_log  /var/log/nginx/access.log combined if=$loggable;
    ...
}

This configuration will exclude requests that match the regular expression ~*^/admin from being logged.

Customizing log formats

By default, Nginx uses the combined log format, which includes the client IP address, request time, request method, request URL, HTTP version, status code, size of the response, referrer, and user agent. However, you can customize the log format to include only the information that you need.

To customize the log format, modify the access_log directive in the Nginx configuration file. For example, to include only the client IP address, request time, request URL, and user agent, add the following line to the http block in the Nginx configuration file:

log_format  mylog  '$remote_addr - $time_local - "$request" - "$http_user_agent"';

Then, update the access_log directive to use the new log format:

access_log  /var/log/nginx/access.log mylog;

This will create a log file at /var/log/nginx/access.log using the mylog log format.

Conclusion

In this guide, we have shown you how to configure logging and log rotation in Nginx on an Ubuntu 22 and Debian Server 12. By following these steps, you can ensure that your Nginx logs are properly configured and rotated to prevent disk space issues and make it easier to analyze the logs when troubleshooting issues. We have also provided some tips and tricks to help you monitor and analyze the logs, customize log formats, and exclude specific requests from being logged.

How to upgrade old version of Nginx on Ubuntu 22.04

Mayur Chavhan — Wed, 26 Jul 2023 11:46:00 GMT

Nginx is a powerful web server, load balancer, and reverse proxy that is used by some of the most popular websites in the world. It can help improve the performance and security of your web applications, and this guide will show you how to install the latest version of Nginx on Ubuntu 22.04.

To install Nginx, you need to follow these steps:

Log in as root To proceed with the installation of Nginx, you need to be logged in as root. If you are not already logged in as root, you can switch to the root user using the following command:

$ sudo -i

Update package list The next step is to update the package list using the following command:

# apt update

Install required packages Install the required packages to your system using the following command:

# apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring -y

Import the Nginx signing key Import the Nginx signing key using the following command:

# wget -O- <https://nginx.org/keys/nginx_signing.key> | gpg --dearmor \\
    | tee /etc/apt/trusted.gpg.d/nginx.gpg > /dev/null

Verify the key Verify that the downloaded file contains the proper key using the following command:

# gpg --dry-run --quiet --import --import-options import-show /etc/apt/trusted.gpg.d/nginx.gpg

Set up the apt repository Set up the apt repository for stable Nginx packages using the following command:

# echo "deb <http://nginx.org/packages/ubuntu> `lsb_release -cs` nginx" \\
    | tee /etc/apt/sources.list.d/nginx.list

Update repository information Update the repository information using the following command:

# apt update

Remove existing Nginx installations Remove all existing Nginx installations using the following command. (This step can be skipped on new systems.)

# apt purge nginx nginx-common nginx-full nginx-core

Install Nginx Install Nginx using the following command:

# apt install nginx

Verify the installation Verify the installation and Nginx version using the following command:

# nginx -v

Enable the Nginx service Enable the Nginx service using the following command:

# systemctl enable nginx

Start Nginx Start Nginx using the following command:

# systemctl start nginx

Modify the default configuration The default configuration when installing Nginx through the Nginx repository differs from the default configuration when installing Nginx through the Ubuntu repository. We will modify a few things to achieve this. First, create additional directories using the following command:

# mkdir /etc/nginx/{modules-available,modules-enabled,sites-available,sites-enabled,snippets}

Edit the nginx.conf file Edit the nginx.conf file using the following command:

# cat > /etc/nginx/nginx.conf <<EOF
user  www-data;
worker_processes  auto;
pid        /var/run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
    worker_connections  1024;
}
http {
    sendfile on;
    tcp_nopush on;
    types_hash_max_size 2048;

    server_tokens off;

    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    access_log  /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    gzip  on;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}
EOF

Check the configuration Check the configuration using the following command:

# nginx -t

Restart Nginx Restart Nginx using the following command:

# systemctl restart nginx

Test Nginx Test if Nginx is responding using the curl command:

# curl localhost

It is important to note that this tutorial assumes you are using Ubuntu 22.04. If you are using a different version of Ubuntu or a different operating system, the commands may be different. Also, make sure you have appropriate permissions before running commands.

How To Automate Initial Server Setup of Multiple Ubuntu 22.04 Servers Using Ansible

Mayur Chavhan — Thu, 13 Jul 2023 21:34:00 GMT

Automation is a key aspect of modern infrastructure management. It allows you to quickly and easily perform repetitive tasks across multiple servers with minimal human intervention. Ansible is a popular automation tool that enables you to automate the initial server setup of multiple Ubuntu 22.04 servers with ease.

In this guide, we will walk you through the steps to automate the initial server setup of multiple Ubuntu 22.04 servers using Ansible. We will cover the installation of Ansible, creating an inventory file, configuring SSH access, setting up sudo access, and installing some common packages.

Prerequisites

Before we begin, you will need the following:

Multiple Ubuntu 22.04 servers.
A user account with sudo privileges on each server.
Ansible installed on your local machine.

Step 1: Installing Ansible

Ansible is not installed by default on Ubuntu 22.04. To install Ansible on your local machine, follow these steps:

Open a terminal window on your local machine.
Update the package lists and install Ansible with the following command:

sudo apt update
sudo apt install ansible

Verify that Ansible is installed by running the following command:

ansible --version

You should see the version of Ansible that you installed.

Step 2: Creating an Inventory File

The inventory file is a list of all the servers that Ansible should manage. This file is written in INI format and can be located anywhere on your local machine. To create an inventory file, follow these steps:

Open a terminal window on your local machine.
Create a new text file with the following command:

nano inventory.ini

Add the IP addresses or hostnames of each server to the file, one per line:

[webserver]
192.168.1.101
192.168.1.102

[database]
192.168.1.103
192.168.1.104

In this example, we have two groups of servers: webserver and database. The IP addresses of the servers in each group are listed below the group name.

Save and close the file.

Step 3: Configuring SSH Access

In order for Ansible to manage your servers, it needs to be able to connect to them using SSH. To configure SSH access, follow these steps:

Generate an SSH key pair on your local machine with the following command:

ssh-keygen

Copy the public key to each server with the following command:

ssh-copy-id username@server_ip_address

Test that you can connect to each server with the following command:

ssh username@server_ip_address

Step 4: Setting Up Sudo Access

In order to perform certain tasks, such as installing packages, Ansible needs to be able to run commands with sudo privileges. To set up sudo access, follow these steps:

Open a terminal window on each server.
Add your user account to the sudo group with the following command:

sudo usermod -aG sudo username

Test that your user account has sudo access with the following command:

sudo whoami

You should see “root” as the output.

Step 5: Installing Common Packages

Now that Ansible is set up and configured to manage your servers, we can install some common packages. To do this, we will create a playbook.

Create a new text file with the following command:

nano playbook.yml

Add the following code to the file:

---
- name: Install common packages
  hosts: all
  become: true
  tasks:
    - name: Update package lists
      apt:
        update_cache: yes

    - name: Install packages
      apt:
        name:
          - nano
          - git
          - curl
          - wget

This playbook will update the package lists and install the Nano, Git, Curl, and Wget packages on all servers in the inventory file.

Save and close the file.
Run the playbook with the following command:

ansible-playbook -i inventory.ini playbook.yml

Ansible will connect to each server, update the package lists, and install the specified packages.

Step 6: Creating a Custom User

By default, Ubuntu 22.04 comes with a user named “ubuntu”. It is recommended that you create a custom user with a unique username and password for security reasons. To create a new user, follow these steps:

Open a terminal window on each server.
Create a new user with the following command, replacing “newuser” with your desired username:

sudo adduser newuser

Set a password for the new user when prompted.
Add the new user to the sudo group with the following command:

sudo usermod -aG sudo newuser

Test that the new user has sudo access with the following command:

sudo whoami

You should see “root” as the output.

Step 7: Configuring Firewall

A firewall is an essential security tool that prevents unauthorized access to your servers. Ubuntu 22.04 comes with UFW (Uncomplicated Firewall) pre-installed. To configure UFW, follow these steps:

Open a terminal window on each server.
Enable UFW with the following command:

sudo ufw enable

Allow SSH access with the following command:

sudo ufw allow ssh

Allow HTTP and HTTPS access (if applicable) with the following command:

sudo ufw allow http
sudo ufw allow https

Verify that the firewall is configured correctly with the following command:

sudo ufw status verbose

You should see the rules that you just configured listed.

Step 8: Configuring Timezone

By default, Ubuntu 22.04 is set to the UTC timezone. To change the timezone, follow these steps:

Open a terminal window on each server.
List the available time zones with the following command:

timedatectl list-timezones

Set the timezone to your desired timezone with the following command, replacing “America/New_York” with your desired timezone:

sudo timedatectl set-timezone America/New_York

Verify that the timezone is set correctly with the following command:

timedatectl

You should see the timezone that you just set listed.

Conclusion

In this guide, we have shown you how to automate the initial server setup of multiple Ubuntu 22.04 servers using Ansible. We covered the installation of Ansible, creating an inventory file, configuring SSH access, setting up sudo access, installing some common packages, creating a custom user, configuring firewall, and configuring timezone. With this knowledge, you can easily automate the setup and configuration of your infrastructure, saving you time and effort.

How to Install AWS CLI on Linux, Windows, and Mac

Mayur Chavhan — Thu, 13 Jul 2023 21:30:00 GMT

AWS CLI (Command Line Interface) is a command-line tool used by developers and system administrators to interact with AWS services. In this tutorial, we will learn how to install AWS CLI on Linux, Windows, and Mac. We will also learn some tips and tricks to make working with AWS CLI easier.

Prerequisites

Before we start, make sure you have the following prerequisites:

A Linux, Windows, or Mac machine with administrative privileges
Python 2.7.9 or later, or Python 3.4 or later

Installing AWS CLI on Linux

Follow the steps below to install AWS CLI on Linux:

Open the terminal on your Linux machine.
Update the package list:
```
sudo apt-get update
```
Install the AWS CLI package:
```
sudo apt-get install awscli
```
Verify the installation:
```
aws --version
```
The above command should output the version of AWS CLI installed on your machine.

Installing AWS CLI on Windows

Follow the steps below to install AWS CLI on Windows:

Download the AWS CLI MSI installer for Windows from the official AWS CLI website.
Run the installer and follow the prompts to install AWS CLI.
Open Command Prompt or PowerShell and run the following command:
```
aws --version
```
The above command should output the version of AWS CLI installed on your machine.

Installing AWS CLI on Mac

Follow the steps below to install AWS CLI on Mac:

Open the terminal on your Mac machine.
Install AWS CLI using Homebrew:
```
brew install awscli
```
Verify the installation:
```
aws --version
```
The above command should output the version of AWS CLI installed on your machine.

Tips and Tricks

Multiple AWS Profiles

You can configure multiple AWS profiles on your machine. This is useful if you have multiple AWS accounts or if you are working with multiple IAM users.

To create a new AWS profile, run the following command:

aws configure --profile <profile-name>

Replace <profile-name> with the name of your new profile. Follow the prompts to provide your AWS access key, secret access key, region, and output format.

To switch between AWS profiles, specify the profile name when running AWS CLI commands:

aws s3 ls --profile <profile-name>

Using AWS SSO

If your organization uses AWS SSO (Single Sign-On), you can use AWS CLI to log in to your AWS account without providing your AWS access key and secret access key.

To use AWS SSO with AWS CLI, run the following command:

aws sso login --profile <profile-name> --region <aws-region>

Replace <profile-name> with the name of your AWS profile and <aws-region> with the AWS region you want to log in to.

Syncing Files to Amazon S3

You can use AWS CLI to sync files and directories to Amazon S3. This is useful for backing up files to Amazon S3 or for distributing files to a large number of users.

To sync a local directory to an S3 bucket, run the following command:

aws s3 sync /path/to/local/directory s3://<bucket-name>/<prefix>

Replace /path/to/local/directory with the path to your local directory, <bucket-name> with the name of your S3 bucket, and <prefix> with the prefix to use for the uploaded files.

Copying Files Between Amazon S3 Buckets

You can use AWS CLI to copy files between Amazon S3 buckets. This is useful if you want to duplicate files in different buckets or regions.

To copy a file between S3 buckets, run the following command:

aws s3 cp s3://<source-bucket>/<source-key> s3://<destination-bucket>/<destination-key>

Replace <source-bucket> with the name of the source S3 bucket, <source-key> with the key of the source file, <destination-bucket> with the name of the destination S3 bucket, and <destination-key> with the key of the destination file.

Conclusion

In this tutorial, we learned how to install AWS CLI on Linux, Windows, and Mac. We also learned some tips and tricks to make working with AWS CLI easier. With AWS CLI, you can manage your AWS infrastructure from the command line, making it easier to automate common tasks.

How to Install MariaDB 11.0 With phpMyAdmin on Rocky / AmaLinux

Mayur Chavhan — Thu, 13 Jul 2023 21:15:00 GMT

MariaDB is an open-source relational database management system that is a drop-in replacement for MySQL. It is developed by the original creators of MySQL and is widely used in web applications. phpMyAdmin is a free and open-source web-based application that provides a graphical user interface for managing MySQL and MariaDB databases. In this tutorial, we will install MariaDB 11.0 and phpMyAdmin on Rocky / AmaLinux.

Prerequisites

Before starting the installation process, make sure that you have the following:

A Rocky / AmaLinux server with sudo privileges.
Access to the internet to download and install the required packages.

Step 1: Update the System

The first step is to update the system packages to their latest versions. Open the terminal and execute the following command:

sudo yum update

This command will update all the installed packages to their latest versions.

Step 2: Install MariaDB 11.0

To install MariaDB 11.0, execute the following command in the terminal:

sudo yum install -y mariadb-server

This command will install the MariaDB server on your system. Once the installation is complete, start the MariaDB service and enable it to start at boot time using the following commands:

sudo systemctl start mariadb
sudo systemctl enable mariadb

Next, run the following command to secure your MariaDB server:

sudo mysql_secure_installation

This command will prompt you to set a root password, remove anonymous users, disallow remote root login, and remove test databases. Follow the prompts and answer the questions to secure your MariaDB server.

Step 3: Install phpMyAdmin

To install phpMyAdmin, execute the following command in the terminal:

sudo yum install -y epel-release
sudo yum install -y phpMyAdmin

This command will install phpMyAdmin and its dependencies on your system.

Step 4: Configure phpMyAdmin

After installing phpMyAdmin, you need to configure it to work with MariaDB. Open the phpMyAdmin configuration file using the following command:

sudo nano /etc/httpd/conf.d/phpMyAdmin.conf

In this file, find the following line:

<IfModule mod_authz_core.c>

Add the following lines after it:

# Apache 2.4
<RequireAny>
Require ip 127.0.0.1
Require ip ::1
</RequireAny>

Save and close the file.

Step 5: Restart the Services

After making the necessary changes to the configuration files, restart the services using the following commands:

sudo systemctl restart httpd
sudo systemctl restart mariadb

Step 6: Access phpMyAdmin

Open your web browser and navigate to the following URL:

<http://your-server-ip/phpMyAdmin>

Replace ‘your-server-ip’ with the IP address of your server. You will be prompted to enter your MariaDB username and password. Enter the credentials and click on the ‘Go’ button to access the phpMyAdmin dashboard.

Issues and Fixes

If you encounter any issues during the installation process, try the following fixes:

If you get the error ‘No package epel-release available’, run the following command and try again:

sudo yum install -y <https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm>

If you get the error ‘Access denied for user ‘root‘@’localhost’ (using password: YES)’, try resetting the MariaDB root password using the following commands:

sudo systemctl stop mariadb
sudo mysqld_safe --skip-grant-tables &
mysql -u root
UPDATE mysql.user SET Password=PASSWORD('new_password') WHERE User='root';
FLUSH PRIVILEGES;
exit;
sudo systemctl start mariadb

If you get the error ‘Cannot connect: invalid settings’, open the phpMyAdmin configuration file using the following command and replace the existing lines with the following:

$cfg['Servers'][$i]['auth_type'] = 'cookie';
$cfg['Servers'][$i]['host'] = 'localhost';
$cfg['Servers'][$i]['compress'] = false;
$cfg['Servers'][$i]['AllowNoPassword'] = false;

Conclusion

In this tutorial, we have shown you how to install MariaDB 11.0 with phpMyAdmin on Rocky / AmaLinux. We have also shown you how to configure phpMyAdmin and access it from a web browser. If you encounter any issues during the installation process, try the available fixes.

How to install Podman Compose on Debian 12 (Bookworm)

Mayur Chavhan — Thu, 13 Jul 2023 21:09:00 GMT

f you are running Debian 12 (Bookworm) and want to use Docker Compose or Podman Compose, here are the steps to install Podman Compose on your system.

Comparing Docker and Podman

Docker and Podman are both container runtimes, but they have some differences in their architecture and features.

Architecture

Docker uses a client-server architecture, where the Docker client communicates with the Docker daemon to manage containers. The Docker daemon runs as a background process on the host machine.

Podman, on the other hand, uses a daemonless architecture, where each container is managed as a separate process on the host machine. This makes Podman more lightweight and secure than Docker.

Features

Docker has a larger ecosystem and more features than Podman, including a wider variety of plugins and tools. However, Podman has some unique features that Docker does not have, such as rootless containers and the ability to run containers without a daemon.

Installing Podman Compose

Add the Podman repository to your system by running the following command:

echo "deb <https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/Debian_12/> /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list

Add the repository key to your system by running the following command:

curl -L <https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/Debian_12/Release.key> | sudo apt-key add -

Update the package list by running the following command:
```
sudo apt-get update
```
Install Podman Compose by running the following command:
```
sudo apt-get install podman-compose
```
Verify Podman Compose installation by running the following command:
```
podman-compose version
```
If the installation was successful, you should see the version number of Podman Compose.

Conclusion

Both Docker and Podman have their strengths and weaknesses, and the choice of which one to use depends on your specific use case and requirements. If you value security and lightweight architecture, Podman may be a better choice for you. If you need a wider variety of tools and plugins, Docker may be the better option.

Automate Linux System Management with Ansible System Roles

Mayur Chavhan — Thu, 13 Jul 2023 21:19:00 GMT

Ansible is an open-source automation tool that allows you to automate tasks across multiple servers. Ansible System Roles are pre-written Ansible playbooks that are designed to automate the installation, configuration, and management of specific services and applications on Linux systems.

By using Ansible System Roles, you can automate the deployment of various applications and services like Apache, MySQL, Nginx, PostgreSQL, and many more. In this guide, we will walk you through the steps to automate Linux systems with Ansible System Roles.

Prerequisites

Before we start, you need to have Ansible installed on your system. You can install Ansible by running the following command:

sudo apt-get install ansible

Step 1: Create a Playbook

The first step in automating Linux systems with Ansible System Roles is to create a playbook. A playbook is a file that contains a set of instructions that Ansible will execute on your servers.

To create a playbook, create a file with a .yml extension and add the following code:

---
- name: Install Apache
  hosts: webservers
  become: true
  roles:
    - geerlingguy.apache

In the above code, we have specified the name of the playbook, the hosts on which the playbook will be executed, and the Ansible System Role that we want to use for installing Apache. In this case, we are using the geerlingguy.apache System Role.

Step 2: Define Hosts

The next step is to define the hosts on which you want to execute the playbook. You can define hosts in the /etc/ansible/hosts file. Open the file with your favorite text editor and add the following code:

[webservers]
server1.example.com
server2.example.com

In the above code, we have defined a group of hosts with the name webservers and added two servers to the group.

Step 3: Execute the Playbook

Now that we have created the playbook and defined the hosts, we can execute the playbook by running the following command:

ansible-playbook playbook.yml

In the above command, playbook.yml is the name of the playbook that we created in Step 1.

Example 1: Install Nginx

To install Nginx using Ansible System Roles, create a playbook with the following code:

---
- name: Install Nginx
  hosts: webservers
  become: true
  roles:
    - geerlingguy.nginx

In the above code, we are using the geerlingguy.nginx System Role to install Nginx.

Example 2: Install MySQL

To install MySQL using Ansible System Roles, create a playbook with the following code:

---
- name: Install MySQL
  hosts: databases
  become: true
  roles:
    - geerlingguy.mysql

In the above code, we are using the geerlingguy.mysql System Role to install MySQL.

Example 3: Install PostgreSQL

To install PostgreSQL using Ansible System Roles, create a playbook with the following code:

---
- name: Install PostgreSQL
  hosts: databases
  become: true
  roles:
    - geerlingguy.postgresql

In the above code, we are using the geerlingguy.postgresql System Role to install PostgreSQL.

Example 4: Install Redis

To install Redis using Ansible System Roles, create a playbook with the following code:

---
- name: Install Redis
  hosts: cacheservers
  become: true
  roles:
    - geerlingguy.redis

In the above code, we are using the geerlingguy.redis System Role to install Redis.

Example 5: Install Docker

To install Docker using Ansible System Roles, create a playbook with the following code:

---
- name: Install Docker
  hosts: dockerservers
  become: true
  roles:
    - geerlingguy.docker

In the above code, we are using the geerlingguy.docker System Role to install Docker.

Conclusion

In this guide, we have shown you how to automate Linux systems with Ansible System Roles. By using Ansible System Roles, you can easily automate the deployment of various applications and services on your Linux servers. With the examples provided in this guide, you can now start automating your Linux systems with Ansible System Roles.

How to install PM2 (Process Management) on Ubuntu 22.10 / Debian 11

Mayur Chavhan — Fri, 19 May 2023 06:23:00 GMT

PM2 is a fantastic process manager designed specifically for Node.js applications.

Throughout my journey as a web developer, I've worked extensively with Node.js. I've had to schedule and manage a multitude of Node.js applications using CRON, which, let me tell you, was no walk in the park. That was until I discovered PM2! It was a game-changer, making my life significantly easier. PM2 ensures my apps are always up and running, and it automatically refreshes them whenever I make updates. Plus, it gives me the flexibility to manually set the reload time using CRON or adjust the restart delay for any application. The best part? It works seamlessly across all operating systems!

Now, you might be thinking, "Is PM2 only for Node.js apps?" The answer is a resounding no! While PM2 was indeed created with Node.js applications in mind, its utility isn't confined to just that. After using PM2 for a while, I discovered that it can manage scripts from any programming language! I gave it a whirl with Python, and guess what? It worked like a charm!

In this article, I'm going to share a practical example of how you can schedule and automate your Python scripts using PM2. So, buckle up and let's dive in!

Step 1: Update Your System

First things first, let's make sure your system is up-to-date. Open up your terminal and type in the following command:

sudo apt update
This command will fetch the list of available updates and then upgrade your system. The -y flag automatically confirms all prompts, saving you from having to manually approve each update.

Step 2: Install Node.js

PM2 is a Node.js application, so we'll need to have Node.js installed on our system. Here's how to do it:

sudo apt install nodejs -y
Once the installation is complete, you can verify it by checking the version of Node.js:

nodejs -v
You should see a version number as the output, which means Node.js is installed correctly.

Step 3: Install NPM (Node Package Manager)

NPM is the default package manager for Node.js and it's what we'll use to install PM2. To install NPM, use the following command:

sudo apt install npm -y
Just like we did with Node.js, we can check if NPM is installed correctly:

npm -v
If you see a version number, you're good to go!

Step 4: Install PM2

Now that we have Node.js and NPM installed, we can finally install PM2. Here's the command to do it:

sudo npm install -g pm2
The -g flag installs PM2 globally, which means you can use it from any directory on your system.

To check if PM2 is installed correctly, you can use the following command:

pm2 -v
If you see a version number, congratulations! You've successfully installed PM2 on your Ubuntu 22.10 or Debian 11 system.

Step 5: Set PM2 to Start on Boot

One of the great things about PM2 is that it can automatically restart your applications if your system reboots. To set this up, you can use the following command:

pm2 startup
This command will generate a command that you need to run with superuser privileges. Copy the outputted command and run it:

sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u yourusername --hp /home/yourusername
Remember to replace yourusername with your actual username.

And there you have it! You've installed PM2 on your Ubuntu 22.10 or Debian 11 system. Now you can use PM2 to manage and keep your Node.js, Python or Any script or applications running in the background.

Thank you for reading.

How to bypass specific website or IP from OpenVPN connection on Mac

Mayur Chavhan — Fri, 21 Apr 2023 14:02:00 GMT

We all have sites that needs to bypass the VPN connection and OpenVPN is popular open source VPN software but it lacks to have bypass a website from it’s network but if you’ve a mac then it’s a easy process as shown in below guide and hope it helps you.

To bypass specific websites or IP addresses from OpenVPN connection on a Mac, you can use the route command to exclude them from the VPN tunnel. Here’s how you can do it:

Open the Terminal app on your Mac. You can find it in Applications > Utilities.
First, find your default gateway IP address. You can do this by running the following command in Terminal:

netstat -nr | grep default

This command will display your default gateway IP address in the second column. Note it down as you’ll need it in the next steps.

Now, find the IP address of the website you want to bypass. You can use the following command, replacing “example.com” with the website you want to bypass:

nslookup example.com

This command will give you the IP address of the website. Note it down as you’ll need it in the next steps.

To bypass the website from theOpenVPN connection, use the route command with the following syntax, replacing “your_gateway_ip” with the default gateway IP you noted down earlier, and “website_ip” with the IP address of the website you want to bypass:

sudo route add -host website_ip your_gateway_ip

For example, if your default gateway IP is 192.168.1.1 and the website’s IP is 123.45.67.89, the command would be:

sudo route add -host 123.45.67.89 192.168.1.1

You will be prompted to enter your Mac’s admin password to execute the command. Once you’ve entered the password, the route will be added, and the specified website will be bypassed from the OpenVPN connection.
To check if the bypass is working, you can visit the website and check your IP address. It should show your regular IP address instead of the VPN IP address.
If you want to remove the bypass, you can use the following command, replacing “website_ip” with the IP address of the website you want to remove the bypass for:

sudo route delete -host website_ip

You can repeat steps 3-7 for any additional websites or IP addresses you want to bypass from the OpenVPN connection.

Note: These changes are temporary and will be reset when you restart your Mac. To make the bypass permanent, you can create a script to run these commands at startup or add them to your OpenVPN configuration file.

Streamline Your Container Updates with Docker: A Step-by-Step Guide

Mayur Chavhan — Thu, 30 Jun 2022 21:55:00 GMT

Updating your Docker containers can be a tedious and time-consuming process, especially if you have multiple containers running simultaneously. However, with the right tools and strategies in place, it's possible to streamline this process and make it much more efficient.

Here are some steps you can follow to easily update your Docker containers:

Identify the containers that need to be updated: Before you can update your containers, you'll need to identify which ones need updating. This can be done by running the "docker ps" command and looking for containers with outdated images.
Pull the updated images: Once you've identified the containers that need updating, you can use the "docker pull" command to retrieve the updated images. This will ensure that you have the latest version of the image available.
Stop and remove the old containers: To update a container, you'll need to stop and remove the old one first. You can do this with the "docker stop" and "docker rm" commands.
Run the updated container: Once the old container has been stopped and removed, you can use the "docker run" command to start the updated container. Make sure to specify the updated image name and any necessary environment variables or command-line arguments.
Repeat the process for all necessary containers: If you have multiple containers that need updating, simply repeat these steps for each one.

You can use following commands as an examples:

Pull the updated images for your containers from the Docker registry. You can do this by running the following command:

docker-compose pull

Stop and remove the existing containers. You can do this by running the following command:

docker-compose down

Recreate the containers using the updated images. You can do this by running the following command:

docker-compose up -d

This will update the containers with the latest versions of the images and recreate them with the updated configuration.

Alternatively, you can use the --force-recreate flag with the up command to force the recreation of the containers, even if their configuration has not changed.

docker-compose up -d --force-recreate

By following these steps, you can easily update your Docker containers and ensure that you're running the latest and most secure versions. Happy containerizing!

How to launch Traefik reverse proxy using docker-compose?

Mayur Chavhan — Wed, 29 Jun 2022 16:47:00 GMT

There is a growing demand for centralised, multi-domain, and secure application hosting in light of the proliferation of reverse proxy services. This is partly due to the accessibility of numerous open source projects that encourage experimentation.

Therefore, Traefik is introduced as a reverse proxy, and although there is much discussion about how easy it is to use, most users will find the documentation to be difficult to understand. Even though I am one of them, I have found a method to overcome my situation, and I hope to teach others to do the same.

I'm using Traefik as a reverse proxy to publish and secure services that are running in Docker containers because that's where most of the popular open source projects are housed. In this blog post, I'll show you how to use docker-compose to deploy Traefik 2 across many hosts on a local machine.

If you're using Docker and want to utilize Traefik as a reverse proxy, you may use docker-compose to set up the Traefik container and any additional containers you'd like to run behind the proxy.

Here is an example docker-compose.yml file that sets up Traefik as a reverse proxy:

version: '3'

services:
  traefik:
    image: traefik:latest
    command: --api --docker
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.toml:/etc/traefik/traefik.toml
      - ./acme.json:/acme.json
    networks:
      - web

networks:
  web:
    driver: bridge

This docker-compose.yml file does the following:

Defines a single service, named traefik, that runs the latest version of the Traefik Docker image.
Exposes the Traefik dashboard on port 8080 and the default HTTP port (80) to the host machine.
Mounts the Docker socket file as a volume, so that Traefik can listen for container events and update its configuration accordingly.
Mounts a configuration file (traefik.toml) and an ACME JSON file (acme.json), which are used to configure Traefik's behavior and store SSL certificates, respectively.
Creates a custom network named web and assigns the traefik service to it.

To start Traefik and any other containers behind the proxy, run the following command:

$ docker-compose up -d

This will start the traefik service in the background (detached mode) and create any other necessary containers, as specified in the docker-compose.yml file.

Note: The specific configuration options used in the traefik.toml file will depend on your specific needs and setup. You can find more information about how to configure Traefik in the Traefik documentation.

How to secure your site from SQL Injection, Exploits and Spamming Agents using Nginx.

Mayur Chavhan — Mon, 19 Jul 2021 19:37:00 GMT

Nginx is one of most popular web server that has so many features that even it may surprise you. One of best feature of Nginx that it has huge library of security policies that it makes your web-server absolute hacker-proof and you won't even need another tool to protect your sites.

All your Nginx sites config can be import from another location so for clean code we are going to keep the security policies on a separate configuration file.

Lets call our file a security.conf under "/etc/nginx/"

There are variant of attacks can be done by attackers and famous attack types are listed down below,

SQL Injection
File Injection
Spam
User Agents
Bandwidth Hogs and Hacking Tools

Here's the security.conf file content as follows,

$ cd /etc/nginx

$ nano security.conf

Paste below code into the file and save the file using CTRL + X and ENTER.

##
# Protection against SQL injection
##
location ~* "(eval()"  { deny all; }
location ~* "(127.0.0.1)"  { deny all; }
location ~* "([a-z0-9]{2000})" { deny all; }
location ~* "(javascript:)(.*)(;)"  { deny all; }
location ~* "(base64_encode)(.*)(()"  { deny all; }
location ~* "(GLOBALS|REQUEST)(=|[|%)"  { deny all; }
location ~* "(<|%3C).*script.*(>|%3)" { deny all; }
location ~ "(|...|../|~|`|<|>||)" { deny all; }
location ~* "(boot.ini|etc/passwd|self/environ)" { deny all; }
location ~* "(thumbs?(_editor|open)?|tim(thumb)?).php" { deny all; }
location ~* "('|")(.*)(drop|insert|md5|select|union)" { deny all; }
location ~* "(https?|ftp|php):/" { deny all; }
location ~* "(='|=%27|/'/?)." { deny all; }
location ~ "({0}|(/(|...|+++|"")" { deny all; }
location ~ "(~|`|<|>|:|;|%||s|{|}|[|]||)" { deny all; }
location ~* "/(=|$&|_mm|(wp-)?config.|cgi-|etc/passwd|muieblack)" { deny all; }
location ~* "(&pws=0|_vti_|(null)|{$itemURL}|echo(.*)kae|etc/passwd|eval(|self/environ)" { deny all; }
location ~* ".(aspx?|bash|bak?|cfg|cgi|dll|exe|git|hg|ini|jsp|log|mdb|out|sql|svn|swp|tar|rdf)$" { deny all; }
location ~* "/(^$|mobiquo|phpinfo|shell|sqlpatch|thumb|thumb_editor|thumbopen|timthumb|webshell).php" { deny all; }
##
# Block SQL injections Attacks
##
set $block_sql_injections 0; if ($query_string ~ "union.*select.*(") { set $block_sql_injections 1; }
if ($query_string ~ "union.*all.*select.*") { set $block_sql_injections 1; }
if ($query_string ~ "concat.*(") { set $block_sql_injections 1; }
if ($block_sql_injections = 1) { return 404; }
##
# Block File injections Attacks
##
set $block_file_injections 0;
if ($query_string ~ "[a-zA-Z0-9_]=http://") { set $block_file_injections 1; }
if ($query_string ~ "[a-zA-Z0-9_]=(..//?)+") { set $block_file_injections 1; }
if ($query_string ~ "[a-zA-Z0-9_]=/([a-z0-9_.]//?)+") { set $block_file_injections 1; }
if ($block_file_injections = 1) { return 404; }
##
# Block common bad exploits 
##
set $block_common_exploits 0;
if ($query_string ~ "(<|%3C).*script.*(>|%3E)") { set $block_common_exploits 1; }
if ($query_string ~ "GLOBALS(=|[|%[0-9A-Z]{0,2})") { set $block_common_exploits 1; }
if ($query_string ~ "_REQUEST(=|[|%[0-9A-Z]{0,2})") { set $block_common_exploits 1; }
if ($query_string ~ "proc/self/environ") { set $block_common_exploits 1; }
if ($query_string ~ "mosConfig_[a-zA-Z_]{1,21}(=|%3D)") { set $block_common_exploits 1; }
if ($query_string ~ "base64_(en|de)code(.*)") { set $block_common_exploits 1; }
if ($block_common_exploits = 1) { return 404; }
##
# Block SPAM Keywords
##
set $block_spam 0;
if ($query_string ~ "b(ultram|unicauca|valium|viagra|vicodin|xanax|ypxaieo)b") { set $block_spam 1; }
if ($query_string ~ "b(erections|hoodia|huronriveracres|impotence|levitra|libido)b") { set $block_spam 1; }
if ($query_string ~ "b(ambien|bluespill|cialis|cocaine|ejaculation|erectile)b") { set $block_spam 1; }
if ($query_string ~ "b(lipitor|phentermin|pro[sz]ac|sandyauer|tramadol|troyhamby)b") { set $block_spam 1; }
if ($block_spam = 1) { return 404; }
##
# Block bad user agents
##
set $block_user_agents 0;
# Don't disable wget if you need it to run cron jobs!
#if ($http_user_agent ~ "Wget") { set $block_user_agents 1; }
# Disable Akeeba Remote Control 2.5 and earlier
if ($http_user_agent ~ "Indy Library") { set $block_user_agents 1; }
##
# Common bandwidth hoggers and hacking tools.
##
if ($http_user_agent ~ "libwww-perl") { set $block_user_agents 1; }
if ($http_user_agent ~ "GetRight") { set $block_user_agents 1; }
if ($http_user_agent ~ "GetWeb!") { set $block_user_agents 1; }
if ($http_user_agent ~ "Go!Zilla") { set $block_user_agents 1; }
if ($http_user_agent ~ "Download Demon") { set $block_user_agents 1; }
if ($http_user_agent ~ "Go-Ahead-Got-It") { set $block_user_agents 1; }
if ($http_user_agent ~ "TurnitinBot") { set $block_user_agents 1; }
if ($http_user_agent ~ "GrabNet") { set $block_user_agents 1; }
if ($block_user_agents = 1) { return 404; }
##

Now, Go to the site-available folder and add the below line to running site configuration file to secure the site,

For example, If you wish to protect the www.example.com.conf then edit the file and add before closing " } "

 include security.conf;

Since, security.conf file exists at /etc/nginx folder root path so you don't have to put whole file path.

After these changes you will need to reload nginx configuration if everything is in order but before that verify Nginx configuration is working properly following this command,

$ sudo nginx -t

if output of above command shows this then it's mean all went OK...

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Next step is to reload Nginx service to see the effect of configuration we made in the website.

$ service nginx reload

Voila!! You're SET!!

How to create Swap Partition on Ubuntu 20.04 and Debian 10

Mayur Chavhan — Mon, 19 Jul 2021 11:37:00 GMT

Create a File
Convert file into SWAP compatible file.
Mount SWAP File
Add SWAP at system boot entry.

On AWS, If you are using smaller instance type like t2.small and t3.small or any Digital Ocean instance you are using which requires more RAM resources then you know it won't be enough for your application and it's application performance will take hit.

Since, We have SSD / NVMe Storage solutions which are very fast can be use as an alternative to operate as a RAM so we can create a SWAP partition which is basically tells Operating System to store CPU operation data on the disk partition when RAM space is not sufficient but SWAP partition is not fast as traditional RAM but it gets the job done with very little work.

This article specifically made to help you to enable SWAP Partition to mount on the system for your instance or VPS. Any Ubuntu or Debian based Linux operating system has same commands to set SWAP partition so you this guide will be useful for any debian based OS also for Ubuntu Desktop Users as well.

Create a File

Consider, You got an instance or VPS of 2 vCore and 2 GB RAM.

Use below command to create a 1 GB of Swap,

$ sudo dd if=/dev/zero of=/var/swapfile bs=1M count=1024

Above command will create a file "swapfile" in "/var" folder, but also learn what's the command actually doing,

bs=1M count=1024 will create 1GB "swapfile", so you can change count if you want to use count as 2048, 4096 for 2GB and 4GB respectively.

Alternative, Similar command you can use to create swapfile ,

$ sudo fallocate -l 1G /var/swapfile

Convert file into SWAP compatible file.

Now, Let's convert this file into an actual swap,

$ sudo mkswap /var/swapfile

We almost done,

Mount SWAP File

$ sudo swapon /var/swapfile

Now we are done, We have mounted an swapfile into swap partition so CPU will use it if RAM storage is nearly full.

You can check if swap partition properly mounted or not by using below command,

$ sudo swapon --show

Add SWAP at system boot entry.

At times, When VPS or instance will get rebooted or at system boot, Swap partition will get unmounted if it's entry not added in system boot entry.

Just add an entry to "/etc/fstab" file, Which System check if any partition needs to be mounted at boot.

$ echo "/var/swapfile swap swap defaults 0 0" | sudo tee -a /etc/fstab

That's it. You've just upgraded your server with a virtual RAM.

If you feel like a geek and really want to see summary of swap partition,

$ swapon --summary

Automate Smart Plug to charge laptop using Webhook and Shell Script.

Mayur Chavhan — Sun, 06 Jun 2021 19:47:00 GMT

In the world of automation, it little gives me heebie-jeebies to turn on and off charging when needed cause I read that always plugged in charging will damage your laptop battery but since I'm using a laptop it always draws power so keeping on charging all the time is good or bad that is debate we don't want to get in, so here we go..

Get a SMART Switch
Create an account on Sequematic
Write or Download Bash Script for an automation.
Cronjob for scheduled task run.

ASUS Laptop where they have inbuilt a hardware charging limit to 80% so I thought that it must be important that's why ASUS has done it. But now, the office gave me another working laptop which is DELL, and sadly given the model doesn't have a hardware charging limit available and while it's plugged and charging at full, sometimes laptop gets hot and its bad if you live in tropical area like myself. 😩

Anyhow, You can see limiting the charging can increase your laptop's battery life a little better.

So here I have a problem so I thought, What if I can build a simple plug as a charger solution that can be turned automatically ON when charging gets low or reaches a set limit and it will get off when laptop charging reaches above 90%. and vice versa. So it a simple and fun to build this want to share this little project. 😎

Get a SMART Switch

The first step is to get a wireless plug so I found the cheapest smart plug available on Amazon. which advertised as works with Alexa so without any research I thought it must work with everything. Boy! I was wrong but eventually, it worked out just fine, just went to my super helpful everyone's favourite friend who helped me to fix this kind of issue as always and you all know by its name "google". 🤓

So this Solimo Smart Plug has this app called "Smart Life" to set up your device with your local centralized Wi-Fi router so you can talk to this "IOT" device from anywhere.

https://amzn.to/354UpQW

So I register myself on the app and proceed with usage turning ON and OFF. Also, This app has multi-functionality like I can set it up like a timer but sadly I have to set up the timer again and again like Microwave or create a scene where I can connect to other services like location, temperature, etc. which is all fine but it's useless for the purpose and what I want to achieve.

Create an account on Sequematic

If you are familiar with IFTTT it's simply "IF THIS THEN THAT" and my project is all about that so I wanted to find an option to turn the plug state from my laptop whenever charging is necessary. Then I found out that I can turn on the device through Webhook, where I change the plug state as I want and the webhook is available on IFTTT. Then I searched in IFTTT if there's an applet available for "Smart Life" and found some applets.

Great !! but then the PROBLEM occurred, where it showed my plug is unavailable in the applet settings so I search about it and found "Smart Life" broke ties with IFTTT so I search more than found out about another app called "Brilliant Smart" and really it's not that brilliant which turns out the same
Table of Contents
1. Get a SMART Switch
2. Create an account on Sequematic
3. Write or Download Bash Script for an automation.
4. Cronjob for scheduled task run.
Table of Contents
1. Get a SMART Switch
2. Create an account on Sequematic
3. Write or Download Bash Script for an automation.
4. Cronjob for scheduled task run.

failure as "Smart Life" then I went again for help on the internet and finally someone on Reddit said that there's another website which still can access "Smart Life" API and that site is "sequematic" which finally done all the work for me.

Sequematic is really easy to set up so I created a step by creating a webhook where I gave the state name as a "turn_on" and set parameter "on" and another step I have added a switch to turn on whenever a POST request on the web-hook is being sent.

In the above screenshot, you can see two sequences are created which are "socket-switch-on" and "socket-switch-off". In those sequences, two steps have been created.

Cool !! Now my plug can be turned on from the internet using just a POST request. It was so simple so what now? How can I turn it on whenever my laptop needs charging?

Write or Download Bash Script for an automation.

As always I wrote a very simple bash script [ Literally, It is very simple ], Check it out.

#!/bin/bash # get battery percentage battery_current_charge=$(upower -i $(upower -e | grep '/battery') | grep --color=never -E percentage|xargs|cut -d' ' -f2|sed s/%//) # check if battery percentage with your set value, mine is 40 if [[ $battery_current_charge -le "40" ]]; then # if you using Ubuntu then below notify command will show you notification notify-send "Auto Charging" "Script Executed!" #This will trigger switch on if condition is satisfied curl -X POST https://sequematic.com/trigger-ifttt-webhook/your-id//switch_on >/dev/null 2>&1 fi if [[ $battery_current_charge -ge "90" ]]; then # This condition will turn off the charging when limit hit to 90 curl -X POST https://sequematic.com/trigger-ifttt-webhook/your-id//switch_off >/dev/null 2>&1 fi

So if you have some Linux experience then you can understand from the shell script that I have created two if conditions and in the first condition, it compares the current battery percentage to start the plug when the battery reaches below or equal to 40%.

The second condition when the battery percentage reaches above or equal to 90% smart plug should automatically turn off.

In the shell script, the URL taken from sequematic can change the plug state when the condition is satisfied so you need to replace both of your IFTTT URLs from the sequematic link in the script.

Cronjob for scheduled task run.

This is a simple task given in a simple script but how can this script know every time this battery value changes?

As always, Linux has inbuilt tools like Cron, Where, I have given a timer to run this script for every 5 min to check battery percentage and the script will check its logic.

Check out the below cronjob.

*/5 * * * * bash charge-battery-script.sh

Voila!! It worked like magic.

I always use Ubuntu as my daily driver OS so it was a piece of cake for me to automate this process and if you want to do this on your Windows then, I suggest you should turn on WSL Feature "Windows Subsystem for Linux" which of course enables Linux (Ubuntu) in your windows.

I can think of many usages from this automation somethings like turn on smart lights or change the color of smart light, smart socket or plug activity from your laptop.

Hope you found this article interesting and let us know your opinion if you have a way to improve it.

Thank you for reading!

How to install Portainer on Linux OS (Ubuntu, Debian, CentOS, Raspbian)

Mayur Chavhan — Mon, 15 Jun 2020 07:33:00 GMT

Overview:

Portainer is docker web GUI application in where you can manage all your docker containers, Images and Volumes. It offers more features than just showing docker containers but you can even manage other docker nodes in one Portainer application.

This guide will show you how easy is to install Portainer on any linux machines but before that you need to keep in mind that it needs some preparation before installing it.

There are two method is to install Portainer but this guide will show you the easiest one.

Installation.

The easiest method to install Portainer is of course is docker. To manage Docker, you need to deploy a docker container that contains Portainer binary and it access host docker process file to get all information what are docker containers with volumes are running.

Check out our guide on how to install Docker on Linux so you can install Portainer.

Now, Here we go, To Install Portainer here is this one line command that simply pulls up the Portainer image.

sudo docker run --name portainer --restart=unless-stopped -d -p 8000:8000 -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

In this line of command, We specified docker name "portainer" and port in which Portainer Docker can be accessible to the public network. Which in this case is port "9000"

"-v portainer_data" is host location where on the Docker Host the Portainer data.
"-v /var/run/docker.sock" is Docker process where portainer can access all information and control docker applications.
"portainer/portainer" is the official docker image which will get pull from official Docker repository.

Acess Portainer GUI

Now, Go to the browser and put Docker Host Server IP in there like this:

http://[public-ip-address]:9000

Replace your public ip address in above field and if you are running docker in your private machine then put your private IP like : 192.168.0.10:9000,192.168.1.10:9000 and so on..

Portainer Docker Process

To check if portainer is running on your host machine simply run below command,

sudo docker ps

Command Output:

docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9a83bdecfc56 portainer/portainer "/portainer" 2 hours ago Up 2 hours 0.0.0.0:8000-8000/tcp, 0.0.0.0:9000->9000/tcp portainer

Above output it shows that it exposes application to port 8000 and port 9000 but we use 9000 to access Portainer GUI.

Update Portainer

If you want to update Docker of Portainer then simple run these following commands,

sudo docker pull portainer/portainer

Above command will pull the latest image from official docker repository but it will not update the current running Portainer Docker so we will stop the running container so it will use updated image.

sudo docker stop portainer

Here we have stopped our Portainer container and if you seem to notice we don't have to use docker id to stop portainer container but we used "portainer" name and this happened because we gave our docker container a name called "portainer" so using a name for container is makes easy to manage like stop and run the container.

Now, Delete old Portainer image to replace updated one.

sudo docker rm portainer

Above command will delete only image but not the volume where we are keeping our Portainer data.

Re-run the docker portainer command to launch portainer with updated image at same volume and exposed ports we have given before.

sudo docker run --name portainer --restart=unless-stopped -d -p 8000:8000 -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

Check again for docker process and if it's running then we are good to know. Hope this guide helped you.

Thank you for reading!

Mayur Chavhan

How to install "Nginx Proxy Manager" on Raspberry Pi 4 [arm64]

Mayur Chavhan — Fri, 12 Jun 2020 19:33:00 GMT

Overview:
Prerequisite:
Step-1: Create folders.
Step-2: Create a file called "config.json"
Step-3: Create Docker-Compose File
Step-5: Run Docker-Compose File
Step-6: Login to NPMan dashboard

Overview:

In DevOps deployments Reverse Proxy is very common for port routing in backend applications. One of the popular is Traefik. Similar to Traefik there are HAproxy and Nginx.

HAproxy which is known for Server Load Balancer but also can be used as a reverse proxy.

Nginx is the fastest web server that can also be used as a reverse proxy for backend applications.

All these three are excellent and popular reverse proxies used in servers but all these have one common problem: they need to write configuration files and it's not noobie server admin friendly since it involves years of experience to understand how exactly reverse proxy works. In Nginx and HAproxy, When you need to install SSL certificates for domai, It makes the configuration a little complicated and tiresome where in Traefik, It automatically handles domain certificates which makes Traefik awesome.

When i started learning Traefik it took few months for me to get a proper grip on its configuration and i learned on old Traefik version and when new Traefik versions came out then i have to read their documentation for new version changes plus then i have to added new changes to docker-compose file so my docker container can properly communicate with Traefik routing which adds big sigh for me. Don't get me wrong Traefik is still best for reverse proxy and if you can write Bash and Ansible Yaml scripts to automate with docker or backend applications then you're golden.Where "Jamie Curnow" aka "jc21" did all these three things in a single docker GUI application called "Nginx Proxy Manager '' In which you can easily bind backend applications without writing a single line of code. Which makes it easy enough for newbie server admin also for Developers who don’t need to learn reverse proxy complex configurations.

This installation can run same for any linux distros out there also docker image file also supports architecture for armhf and arm64 so it can easily run on Raspberry Pi 3B and 4B without any issue. I personally used Raspbian Lite for testing purpose and it works well.

Nginx-Proxy-Manager

Prerequisite:

Docker
Docker-Compose
[Optional] Portainer

Step-1: Create folders.

sudo mkdir npman

Inside NPman folder create these three folders [data, data/mysql and letsencrypt] or run below command,

sudo mkdir -p npman/{data/mysql,letsencrypt}

Step-2: Create a file called "config.json"

paste below lines of code into it.

{
  "database": {
    "engine": "mysql",
    "host": "db",
    "name": "DBNAME",
    "user": "DBUSERNAME",
    "password": "DBPASSWORD",
    "port": 3306
  }

Step-3: Create Docker-Compose File

Now, Create a file called "docker-compose.yaml" and paste following code into it.

Source: https://github.com/jeff89179 [ Thanks to him]

version: '2'
services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: always
    ports:
      - '80:80'
      - '443:443'
      - '81:81'
    volumes:
      - './config.json:/app/config/production.json'
      - './data:/data'
      - './letsencrypt:/etc/letsencrypt'
    depends_on:
      - db
  db:
    image: 'yobasystems/alpine-mariadb:latest'
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: npman
      MYSQL_DATABASE: npman
      MYSQL_USER: npman
      MYSQL_PASSWORD: password
    volumes:
      - './data/mysql:/var/lib/mysql'

Change MySQL password for production security purpose.

Step-5: Run Docker-Compose File

Run Docker-Compose command to download docker images and services,

sudo docker-compose up -d

If you have Portainer running then go into the Portainer UI, And in the Stacks, add a new stack named "NPMan" and paste above docker-compose code and deploy it.

After deployment check if container health and port are showing like this..

Step-6: Login to NPMan dashboard

Go to browser and replace <> to docker host server ip like this,

http://<>:81/login

If everything goes right then enter below default credentials,

Credential for NPMan,

Email: admin@example.com

Password: changeme

Visit our repository has all codes in this tutorial,

https://github.com/mayur-chavhan/Nginx-Proxy-Manager

Cheers!

Thank you for reading

How to install Ansible on Ubuntu 18.04 / Debian 10 Buster

Mayur Chavhan — Wed, 13 May 2020 22:58:00 GMT

What is Ansible?
Prerequisites :
Before start...
Add official Ansible repository
Update Packages
Install Ansible

What is Ansible?

Ansible is an open source DevOps automation tool that solves server provisioning in installing tools, configuration, deployment and provisioning. It leverages SSH to communicate between servers. It uses the playbook which states every steps of automation jobs, and playbook uses a simple language called "YAML".

Prerequisites :

Operating System: Ubuntu 18.04 / Debian 10 Buster

SSH Access : You need remote server SSH access available with Password or Key Access so our ansible server can communicate remotely.

Python: python version 2.7 or above it should be installed.

Before start...

The use of Ansible as a capability of managing your server infrastructure, you want to install the Ansible software program on the machine that will serve as the Ansible manage node.

Ansible’s only real dependency is Python. Once Python is installed, the simplest way to get Ansible running is to use pip, a simple package manager for Python. Don't worry most of the linux distro comes with default Python 2.7 and CentOS 8 comes with built in Python version 3.7.

The easiest way to install Ansible on a Debian or Ubuntu system is to use the official apt package.

Add official Ansible repository

From your ansible host node, run the following command to consist of the authentic project’s PPA and it is needed else you'll end up with older version of Ansible :

sudo apt-add-repository ppa:ansible/ansible

Press ENTER to proceed with PPA addition.

Update Packages

Newly added PPA needs to be updated :

sudo apt update

If an error like “sudo: add-apt-repository: command not found”, then you’re missing the python-software-properties package. Install it with the following command:

sudo apt-get install python-software-properties

Install Ansible

After update is done, you can easily install Ansible using simple install command,

sudo apt-get install ansible -y

Our installation process has been done. Now let’s check ansible version.

sudo ansible --version

That's all folks. Happy Hacking!!

Guide : Git workflow for Developers

Mayur Chavhan — Sun, 17 May 2020 19:46:00 GMT

Overview:
Why Git or Github?
The Workflow Cheat Sheet

Overview:

Git is a top distributed version control system and Nowadays all programmers and developers are dependent on its workflow. Manage to set up a proper Git code workflow is essential for projects that involve group collaboration. Follow procedures to keep working copy of your project through identifying code problems and evolving into a big scale project.

One of the biggest git repository hosting is Github. Git is a distributed version control system accessible through a CLI. Git is free and open-source software distributed under the terms of the GNU General Public License version 2.

Using Github, I can share all of my coding projects with the world. Github has a plethora of unique features that enhance the git experience and make collaborative coding simple. I am far from a Git or Github expert, but this is a short guide to get you started with Git on Github:

Why Git or Github?

It is the best and most popular system available for managing and tracking any code changes also it allows other developers to collaborate on projects at any scale.

In simple words, GitHub is a “hub” for everyone who is working with the Git version control system. It provides a variety of features to make developers' jobs much easier and it's open for the public so anyone can share their code to the development community and can do better projects or make existing code repositories much better by power of collaboration.

System like Git become a lot more obvious once you make sense of how they truly work. The objective of this guide is to reveal some insight into how Git functions in the engine. We're going to investigate a portion of Git's center ideas including its fundamental item stockpiling, how submits work, how branches and labels work, and we'll take a gander at the various types of converging in Git including the much-dreaded rebase. Ideally toward the finish, all things considered, you'll have a strong comprehension of these ideas and will have the option to utilize a portion of Git's further developed highlights with certainty.

It's important now that this guide isn't proposed to be a's first experience with Git. This guide was composed for individuals who as of now use Git, however might want to more readily comprehend it by taking a look in the engine, and get familiar with a couple of slick deceives en route. All things considered, we should start.

This is a short guide to get you started with Git on Github:

| | | | --- | --- | | Repository | Commonly called the “repo,” this is where all of the updated versions (and the primary version) of a project are stored. This is your staging ground for the project. Each repo has a unique URL for easy access. | | Forking | Unless developers pay for a private repository, their project can be viewed by all other members of the GitHub community. If these members feel they could make a worthy contribution, then they can create a new project (repository) from yours. This is called “forking a repo.” | | Pull Request | After forking a repo and making new changes to existing code, GitHub users can invite the developers of the original repository to view the changes. | | Merging | If you like the changes someone made after forking your repository, you can opt to accept those changes and merge them with your existing repository. | | Changelogs | This lets developers working on a single project see all changes, who made them, and when. |

Git workflow is the mixer of steps that we have to follow before starting the development cycle means how developers pull, push, commit, clone and manage code.

The Workflow Cheat Sheet

--------------------------------------------
Managing your Local Repo
--------------------------------------------
NOTE: If you need to hard reset your local repo to match
the remote master use the following commands:
$ git fetch origin
$ git reset --hard origin/master

Undo the act of committing, leaving everything else intact:
$ git reset --soft HEAD^:

Undo the act of committing and everything you'd staged,
but leave the work tree (your files intact):
$ git reset HEAD^

Completely undo it, throwing away all uncommitted changes,
resetting everything to the previous commit:
$ git reset --hard HEAD^

--------------------------------------------
BEGIN WORKFLOW
--------------------------------------------
Clone the Repo to local machine:
$ git clone https://github.com/user_name/repo_name.git

Make sure the local master is up-to-date:
$ git pull origin master

Create new branch:
$ git banch branch_name

Move to branch:
$ git checkout branch_name

Navigate file structure as needed:
$ ls
$ cd folder_name

Add the files to the branch:
$ git add .

Verify file:
$ git status

Commit the files:
$ git commit -m "comment"

Add branch and files to the Remote Repo:
$ git push -u origin branch_name

Go to the github website to manage pull request and merge.

Switch back to local master so you can delete the local branch:
$ git checkout master

Delete local branch:
$ git branch -d branch_name
OR
$ git branch -D branch_name

If you don't want to go to the website, you can merge your branch
to the master locally and push the new master to the remote repo:

Switch back to master branch:
$ git checkout master

Merge the branch with the local master:
$ git merge branch_name -m "comment"

Push the local master to the remote master:
$ git push origin master

Delete local branch:
$ git branch -d branch_name
OR
$ git branch -D branch_name

How to install docker and docker-compose on Ubuntu 18.04

Mayur Chavhan — Wed, 20 May 2020 16:08:00 GMT

Overview
Update System
Install Prerequisite Packages
Add Docker Repositories
Install Docker on Ubuntu 18.04
Start Docker

Overview

Docker Container Image

Docker is an application that makes it simple and easy to run application processes in a container, which are like virtual machines, but more small, more resource-friendly, and more dependent on the host OS Kernel.

Docker-Compose in other hand is more advanced but fairly makes easy if you want to add multiple docker containers and services working together as a one using a single file. YML / YAML is language is used to create docker-compose file.

You’ll learn how to install and use it on an existing installation of Ubuntu 18.04.

Docker requires a 64-bit version of Ubuntu also a kernel version requires to greater than 3.10.X

Ubuntu 18.04 doesn't comes with Docker official repository so we are going to add it and install it. Let's start with process to install Docker and Docker-Compose.

Update System

First and most important is to update the system to keep all packages up to date so we don't run with outdated dependencies.

Run the following commands:

sudo apt update -qq -y

Install Prerequisite Packages

Once we have updated the system, we need to install some necessary packages before we are ready to install Docker. You can do this with the help of a single command:

sudo apt-get install curl apt-transport-https ca-certificates software-properties-common -qq -y

Add Docker Repositories

Following steps is to add Docker Key and Repository to officially supported method for the Docker installation.

Add the GPG key using following command in terminal:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Next, Add the repository:

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

Update the repository information:

sudo apt update -y

Install Docker on Ubuntu 18.04

We are almost close. Use apt command to install Docker:

sudo apt install docker-ce -y

Start Docker

Once the installation is complete, Start the docker to enable the service:

sudo systemctl start docker

Check the service using below command if docker service is active or not.

sudo systemctl status docker

If all goes well then pat yourself in the back. Docker is installed on your Ubuntu machine. Now, Lets install Docker-Compose which is very easy now since we have docker installed on the machine.

Check if you have curl installed in your machine.

If not then install curl using following command:

sudo apt install curl

Install Latest Docker Compose

To download the latest version of Docker Compose, run below command:

sudo curl -L "https://github.com/docker/compose/releases/download/1.25.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

At the time this article was written, Docker Compose 1.25.5 is the latest version but if you want to change latest version go to this release page.

https://github.com/docker/compose/releases

2. Give executable permission to docker-compose at installed location:

sudo chmod +x /usr/local/bin/docker-compose

That's it, Now you've successfully installed docker-compose on your Ubuntu 18.04, Easy isn't it?

How to setup user SSH Keys on Ubuntu 18.04

Mayur Chavhan — Sat, 23 May 2020 11:23:00 GMT

Overview
Step 1 : Generate a RSA Key Pair on your machine.
Step 2 : Copy SSH Public Key to remote server.

Overview

SSH [ Secure Shell ] is a famous encrypted protocol are widely becoming popular for communicate server access. For security purpose SSH provide lot of features and one of the famous is SSH Keys and for some its complicated and mostly biased to use SSH password method.

Cloud Service providers like AWS and GCP are default using ssh key method before creating any Instance. So they can generate SSH keys for you and you can download it for you to access it remotely. But other server provider mostly generate and provide you SSH password.

SSH Password method is secured but password can be bruteforce by millions of bots are out there for cracking passwords and who knows if someone has supercomputer who can break your password within minutes but fear not cause SSH key to the rescue.

In reality, SSH Keys gives an easy and perfectly secure way to connect your remote server and one SSH key can be use for multiple users.

In this tutorial, How we can setup user ssh keys for Ubuntu 18.04 but it also applicable to any linux OS.

For this example we will take a remote server which enabled with SSH password authentication and applying SSH Key based authentication.

Before communication with remote server we have to do following steps.

Step 1 : Generate a RSA Key Pair on your machine.

| | | | --- | --- | | $ | ssh-keygen |

Above command will generate 2048 bit RSA key pair consists of public and private keys. [ id_rsa and id_rsa.pub ] When you run ssh-keygen command it will ask for custom path to store your keys but for now default location is shown in below image.

Press enter to select default path for RSA keys.

Next, You should then see the following prompt:

Enter passphrase (empty for no passphrase):

Additional security if you want to protect your SSH key using passphrase and empty for no passphrase so when you establish remote connection it will not ask for password everytime but its highly recommended that you should use passphrase and it gives more security and that's what we need.

Here's the result after creation of the key

Your identification has been saved in /your_home/.ssh/id_rsa.

Your public key has been saved in /your_home/.ssh/id_rsa.pub.

The key fingerprint is:
SHA256:tQX64BUmMF2BY1wUSTdeJEYQQUjFIf9wrUAz0RJlaU4 user@remote-server
The key's randomart image is:
+---[RSA 2048]----+
| o=+@^#Ooo |
| .BB+XE= |
| .o.*=+ . |
| . = B.. |
| S o o |
| |
| |
| |
| |
+----[SHA256]-----+

And at this path you'll find ssh keys as follows,

That's it, Now you've generated public and private ssh key on your machine. Next step is very easy and we have to copy our public key to remote server so we can SSH Key based authentication for SSH.

Step 2 : Copy SSH Public Key to remote server.

Now we have SSH public key which can authenticate remote machine from our host machine. There is fastest method to copy public key ID to remote host is simple tool called "ssh-copy-id"

Simple yet very useful for copying public ssh key to remote host. Remote host must have enabled password authentication so once public key authentication is done. Later, We can disable password authentication so only gateway to access your server is SSH Key authentication.

ssh-copy-id user@remote_server

[OPTIONAL] - SSH Public Key for Custom Include Key Path

If you have SSH Key available on different path you can use add include path parameter like this

ssh-copy-id -i /ssh-key-folder/sshpub.key user@remote_server

[OPTIONAL] - SSH Public Key for Custom Port

ssh-copy-id -p 5500 user@remote_server

Install Jitsi Meet on Ubuntu 18.04

Mayur Chavhan — Sun, 17 May 2020 20:28:00 GMT

Provisioning

New Ubuntu 18.04 x64 VPS
Create a new sudo user.
A domain jitsi.server.com being pointed to the server instance mentioned above.

Update server packages:

For security and performance purposes, its necessary to update the Ubuntu 18.04 LTS system to the latest status:

sudo apt update sudo apt upgrade -y && sudo shutdown -r now

You need to properly setup a hostname and an FQDN for the machine before you can enable HTTPS security by deploying a Let's Encrypt HTTPS certificate.

The following commands will setup a hostname / FQDN:

sudo hostnamectl set-hostname jitsimeet

sudo sed -i 's/^127.0.1.1.*$/127.0.1.1 jitsi.server.com jitsimeet/g' /etc/hosts

Check if hostname is set properly.

hostname

hostname -f

wget -qO - https://download.jitsi.org/jitsi-key.gpg.key | sudo apt-key add -

apt-get update

echo 'deb https://download.jitsi.org stable/' >> /etc/apt/sources.list.d/jitsi-stable.list

wget -qO - https://download.jitsi.org/jitsi-key.gpg.key | sudo apt-key add -

apt-get install apt-transport-https

apt-get update

apt-get -y install jitsi-meet

sudo /usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh

Firewall rules for Jitsi Meet

As required we need to allow OpenSSH, HTTP, and HTTPS traffic, along with inbound UDP traffic on port range of 5222-5281 and 10000 through port 20000:

sudo ufw allow OpenSSH
sudo ufw allow http
sudo ufw allow https
sudo ufw allow in 4443/tcp
sudo ufw allow in 5222/tcp
sudo ufw allow in 5269/tcp
sudo ufw allow in 5280:5281/tcp
sudo ufw allow in 10000:20000/udp

sudo ufw enable

During the process, input your email admin@gmail.com as prompted and then press ENTER. This script will automatically handle any tasks related to the Let's Encrypt SSL certificate.

Finally, point your favorite web browser to http://jitsimeet.server.com or https://jitsimeet.server.com to access your Jitsi Meet Video conferencing service. Feel free to explore the interface. Clicking the GO button will immediately create a Video conferencing channel for you.