Setting up Flyio volumes & file upload

Persistent File Storage with Fly Volumes

When your Fly.io–hosted application needs to accept and serve user-uploaded files—images, documents, videos, etc.—you require persistent storage outside of the container’s ephemeral filesystem. Fly Volumes provide exactly that: a slice of NVMe disk that survives Machine restarts and deploys, mounted at a directory of your choice on every attached Fly Machine. (fly.io)

---

1. Provisioning Volumes

Start by creating one or more volumes in the same region(s) where your app will run. Use the --count flag to pre-create multiple volumes, which is crucial if you plan to scale to N instances—all volumes must exist so that new instances can attach without errors.

# Create 3 volumes named "uploads" in the iad region, each 5 GB
fly volumes create uploads --size 5 --region iad --count 3

(fly.io)

---

2. Configuring fly.toml

In your fly.toml, add a [mounts] section to your process group (e.g. app) so Fly Machines know where to mount each volume:

app = "my-upload-app"
primary_region = "iad"

[env]
  # Your app’s upload directory environment variable
  UPLOAD_DIR = "/app/uploads"

[mounts]
  source      = "uploads"
  destination = "/app/uploads"

[[services]]
  internal_port = 4000
  protocol      = "tcp"
  # Optional: enforce concurrency so the proxy knows when you’re busy
  [services.concurrency]
    type       = "requests"
    soft_limit = 50
    hard_limit = 60

(fly.io)

---

3. Implementing File Uploads

Within your application code (e.g., Phoenix, Django, Rails), read the UPLOAD_DIR environment variable and save uploads to that path. For example, in a Phoenix style handler:

def render(assigns) do
    ~H"""
    <div class="py-28 lg:px-20 px-4 max-w-5xl mx-auto">
      <.header>
        {@page_title}
        <:subtitle>Use this form to manage blog records in your database.</:subtitle>
      </.header>

      <.form for={@form} id="blog-form" phx-change="validate" phx-submit="save">
        <.input field={@form[:title]} type="text" label="Title" />
        <.input field={@form[:slug]} type="text" label="URL Slug - for SEO" />
        <.input field={@form[:markdown]} type="textarea" label="Markdown" />
        <.input field={@form[:author]} type="text" label="Author" />
        <.input field={@form[:genre]} type="text" label="Genre" />
        <.input field={@form[:author_title]} type="text" label="Author Title" />
        <.input field={@form[:description]} type="text" label="Description" />
        <.input :if={@uploads.thumbnail.entries == []} field={@form[:thumbnail]} type="text" label="Thumbnail" />
        <.button type="button" style="my-5">
          <.live_file_input
            upload={@uploads.thumbnail}
          />
        </.button>
        <div :for={entry <- @uploads.thumbnail.entries} class="flex flex-row my-5">
          <.live_img_preview entry={entry} width={200} />
          <a class="w-5 h-5 cursor-pointer text-3xl ml-5" phx-click="cancel" phx-value-ref={entry.ref}>
            &times;
          </a>
        </div>
        <footer>
          <.button phx-disable-with="Saving..." variant="primary">Save Blog</.button>
          <.button navigate={return_path(@current_scope, @return_to, @blog)}>Cancel</.button>
        </footer>
      </.form>
    </div>
    """
  end

def mount(params, _session, socket) do
    socket =
      allow_upload(socket, :thumbnail,
        accept: ~w(.png .jpg .jpeg),
        max_entries: 1,
        max_file_size: 10_000_000
      )

    {:ok,
     socket
     |> assign(:return_to, return_to(params["return_to"]))
     |> apply_action(socket.assigns.live_action, params)}
  end

defp save_blog(socket, :new, blog_params) do
    thumbnail = get_thumbnail(socket, blog_params["thumbnail"] || "")

    blog_params = Map.put(blog_params, "thumbnail", thumbnail)

    case Blogs.create_blog(socket.assigns.current_scope, blog_params) do
      {:ok, blog} ->
        {:noreply,
         socket
         |> put_flash(:info, "Blog created successfully")
         |> push_navigate(
           to: return_path(socket.assigns.current_scope, socket.assigns.return_to, blog)
         )}

      {:error, %Ecto.Changeset{} = changeset} ->
        {:noreply, assign(socket, form: to_form(changeset))}
    end
  end

  def is_local_image?(url) do
    !String.starts_with?(url, "https")
  end

  def get_thumbnail(socket, existing \\ "") do
    if is_upload?(socket) do
      download_thumbnail(socket)
    else
      existing
    end
  end

def download_thumbnail(socket) do
    consume_uploaded_entries(socket, :thumbnail, fn meta, entry ->
      dest =
        Path.join([
          Application.get_env(:nct_website, :uploads_dir),
          "#{socket.id}#{Path.extname(entry.client_name)}",
        ])
      File.mkdir_p!(Path.dirname(dest))
      File.cp!(meta.path, dest)
      url_path = static_path(socket, "/uploads/#{Path.basename(dest)}")

      {:ok, url_path}
    end) |> List.first()
  end

# In endpoint.ex to serve the /app/uploads to the websites example.com/uploads.
 plug Plug.Static,
    at: "/uploads",
    from: "/app/uploads",
    gzip: false

# In config.exs, add /uploads to this:
config :esbuild,
  version: "0.17.11",
  test: [
    args:
      ~w(js/app.js --bundle --target=es2022 --outdir=../priv/static/assets/js --external:/fonts/* --external:/images/* --external:/uploads/*),
    cd: Path.expand("../assets", __DIR__),
    env: %{"NODE_PATH" => Path.expand("../deps", __DIR__)}
  ],

---

4. Deploy and Verify

fly deploy
fly ssh console --command "ls -l /app/uploads"

Ensure your files persist across deploys and restarts.

---

5. Scaling with Volumes

Manual Horizontal Scaling

When you run fly scale count, Fly will:

1. Attach existing volumes to new Machines, up to the number you pre-created in the region.
2. Create new empty volumes if it runs out of unattached ones.
3. Detach volumes when Machines are destroyed (but leave them intact for reuse).

Note: Data is not copied between volumes—each volume remains independent. (fly.io)

# Scale the "app" process to 3 instances in iad:
fly scale count 3 --region iad

If you didn’t pre-create 3 volumes, fly scale will create the missing volumes—but they’ll be empty. To avoid unexpected empty mounts, always --count at creation time.

---

Automatic Start/Stop (Cost Savings)

For workloads with spiky traffic, you can let Fly Proxy stop idle Machines and start them on demand:

# Inside [[services]] or [http_service]:
auto_stop_machines  = "stop"
auto_start_machines = true
min_machines_running = 1

• auto_stop_machines: when Machines exceed your concurrency soft limit and there’s no traffic, they’re stopped.
• auto_start_machines: on new incoming requests, stopped Machines are started (rather than routing you to a cold path).
• min_machines_running: guarantees at least N running Machines, even when idle. (fly.io)

This setup preserves attached volumes on stopped Machines, so your upload storage remains safe and immediately available upon restart.

---

Metrics-Based Autoscaling

For more fine-grained control—scaling by CPU, queue depth, or custom metrics—you can deploy Fly’s metrics-based autoscaler as a separate app. It polls Prometheus-style metrics and will create or destroy Machines (and handle volume detach/attach) according to your rules. (fly.io)

Best Practice: Pre-create volumes equal to your maximum anticipated instance count to avoid on-the-fly empty volumes.

---

6. Key Considerations

• One-VM Per Volume: Volumes cannot be shared concurrently—each Running Machine needs its own volume.
• No Live Replication: Scaling out does not replicate existing uploads. Use object storage (S3, Upstash Object Storage) or a sync job if you need a shared real-time filesystem.
• Region Placement: Always create volumes in the same region(s) as your Machines to avoid cross-region attachment errors.
• Backups & Snapshots: Use Fly Volume snapshots for backups; restore from snapshots during maintenance windows.

---

Conclusion

Fly Volumes, combined with Fly’s flexible scaling mechanisms—manual, on-demand start/stop, and metrics-based autoscaling—enable you to build resilient, cost-efficient file-upload services. By pre-provisioning volumes, configuring mounts in fly.toml, and choosing the right autoscaling strategy, you can ensure your uploads remain persistent and immediately available, all with minimal DevOps overhead.