Storage Providers

Village Data uses a provider pattern for file storage, enabling swapping between backends.

Interface

interface StorageProvider {
  readonly name: string;
  readonly providerId: string;
  readonly limits: StorageLimits;

  upload(bucket: string, path: string, data: Blob | Buffer | string, options?: UploadOptions): Promise<UploadResult>;
  download(bucket: string, path: string): Promise<DownloadResult>;
  copy(bucket: string, fromPath: string, toPath: string): Promise<CopyResult>;
  delete(bucket: string, paths: string[]): Promise<DeleteResult>;
  list(bucket: string, path: string, options?: { search?: string }): Promise<ListResult>;
  validateFileSize(sizeBytes: number): { valid: boolean; error?: string };
}

Current Implementation

Supabase Provider

• Provider ID: supabase
• Default Limit: 50MB
• Config: SUPABASE_MAX_FILE_SIZE_MB

Usage

Server-Side

import { getStorageProvider, DATASET_FILES_BUCKET } from "@/lib/storage";

const storage = getStorageProvider();

// Upload
const result = await storage.upload(
  DATASET_FILES_BUCKET,
  `datasets/${id}/data.csv`,
  fileBlob,
  { contentType: "text/csv", upsert: true }
);

// Download
const { data } = await storage.download(DATASET_FILES_BUCKET, path);

Client-Side Validation

import { validateFileSize, STORAGE_LIMITS } from "@/lib/storage/limits";

const validation = validateFileSize(file.size);
if (!validation.valid) {
  alert(`Max size: ${STORAGE_LIMITS.maxFileSizeFormatted}`);
}

Provider Tracking

Each dataset_version stores its provider:

storage_provider TEXT DEFAULT 'supabase'
storage_bucket TEXT
storage_path TEXT

This enables migration between providers without breaking existing files.

Adding Providers

1. Implement StorageProvider interface
2. Register in provider factory
3. Update client-side limits if different

See the source code in ui/src/lib/storage/ for implementation details.