Skip to main content
ADAmit Divekar
Back to Blogs
Flask
Python
Cloud Storage
PostgreSQL
Render
Backend Architecture

Building a Custom Cloud Storage Platform with Python, Flask, and PostgreSQL

Amit Divekar

Building a Custom Cloud Storage Platform with Python, Flask, and PostgreSQL

Architecting a cloud storage platform from scratch introduces significant engineering challenges spanning binary data management, recursive schema design, and secure authentication flows. To address these challenges and eliminate dependence on commercial cloud storage providers, I developed FileFlow, a robust web application built using Python, Flask, and PostgreSQL.

This article provides an in-depth analysis of the application's underlying architecture, the database schema design leveraged to support deeply nested directory structures, and the DevOps considerations necessary for deploying stateful file engines to ephemeral cloud infrastructure.

Architectural Overview and Technology Stack

To ensure maintainability and high performance, the technology stack was minimized to industry-standard components:

  • Web Framework: Flask (Python 3.11)
  • Database Layer: PostgreSQL accessed via SQLAlchemy
  • Authentication: Flask-Login combined with Flask-Bcrypt
  • Frontend Interpolation: Jinja2 templating
  • Application Server: Gunicorn WSGI

The system relies heavily on the Flask-SQLAlchemy integration to manage entity relationships and atomic operations reliably.

Modeling Hierarchical Filesystems

The crux of any cloud storage application is effectively mapping a traditional filesystem tree into a relational database. Early prototypes often utilize separate Folder and File tables, leading to complex junction logic and inefficient queries.

In FileFlow, I opted for a Unified Entity Model utilizing a self-referential adjacency list layout.

class File(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    filename = db.Column(db.String(255), nullable=False)
    filepath = db.Column(db.String(500), nullable=False)
    user_id = db.Column(db.Integer, db.ForeignKey('user.id'), nullable=False)
    
    # Hierarchical implementation
    is_folder = db.Column(db.Boolean, default=False)
    parent_folder_id = db.Column(db.Integer, db.ForeignKey('file.id'), nullable=True)
    
    # Metadata fields
    filesize = db.Column(db.Integer, default=0)
    mimetype = db.Column(db.String(100))
    file_hash = db.Column(db.String(64))
    is_favorite = db.Column(db.Boolean, default=False)
    tags = db.Column(db.String(500))

This design decision offers severe distinct advantages:

  1. Simplified Traversals: Because folders and strings share the same database object, computing path resolution and iterating over parent nodes is dramatically simplified.
  2. Global Tagging: Tags and favorite statuses apply universally. A user can favorite an entire nested directory identically to how they would favorite a single PDF document.
  3. Data Integrity Checks: The inclusion of file_hash allows for downstream deduplication processes, dropping server storage overhead by isolating identical cryptographic hashes.

Complex Search and Link Sharing Implementations

Going beyond basic CRUD functionalities, the platform was expanded to include intelligent searching and secure resource sharing.

Search Profiles

Rather than forcing users to rebuild advanced queries, SearchProfile models are persisted in the database. These profiles track query, file_types, size_max, and timestamp boundaries (date_from and date_to). This permits users to instantly execute highly complex filtration patterns via straightforward foreign key lookups.

Ephemeral Link Distribution

File sharing required public endpoints with strict limitations. The ShareLink table generates cryptographically secure, unique tokens mapping back to specific file.id references. By injecting an expires_at column and an access_count tracker, we natively support time-bombed and tightly restricted access URLs, eliminating unauthorized public exposure of binary streams.

Deployment: Conquering Ephemeral Storage on Render

Deploying stateful, file-dependent systems on modern Platform-as-a-Service (PaaS) providers like Render introduces the challenge of ephemeral storage. When an instance sleeps or is redeployed, the local filesystem state resets completely.

To accommodate this without risking data evaporation, the platform utilizes strict process abstractions via an Infrastructure-as-Code render.yaml layout.

services:
  - type: web
    name: fileflow-app
    env: python
    region: oregon
    buildCommand: "./build.sh"
    startCommand: "gunicorn backend.app:app -w 4"
    envVars:
      - key: FLASK_ENV
        value: production

By decoupling the filepath in the actual database from arbitrary physical root directory bindings, migrating from local testing to S3 or a mounted 10GB persistent disk requires adjusting the environment configurations rather than tearing down the database mappings. The PostgreSQL layer retains total contextual awareness regardless of where the binary Blob physically resides.

Final Engineering Thoughts

Constructing FileFlow reinforced the necessity of adhering strictly to relational schema principles and implementing disciplined infrastructure architectures. A web framework handles the HTTP specification effortlessly, but the developer is fundamentally responsible for building secure, scalable state machines.

By centralizing the storage logic, unifying the hierarchy models, and hardening the security boundaries, FileFlow provides a professional-grade alternative to mainstream storage options.

Connect With Me

If you would like to discuss relational architectures or cloud engineering, please reach out directly or open an issue on the public repository.