SFTP compatibility & known limitations

Firepipe speaks standard SFTP in front of cloud object storage, Amazon S3, Azure Blob, Google Cloud Storage, or any S3-compatible store such as Cloudflare R2. Object storage isn't a POSIX filesystem: objects are written whole and have no permission bits or modification-time metadata. The overwhelming majority of SFTP usage, uploading, downloading, listing, deleting, organising into folders, works exactly as you'd expect. A few filesystem-specific behaviours can't be offered, and we'd rather you know them up front than discover them mid-automation. That's what this page is for.

What works well

• Upload, download, resume-friendly transfers, and deletion of files of any size.
• Creating, listing, and removing folders; per-user home directories (path jails).
• Password and SSH public-key authentication; per-user read-only access and quotas.
• The common upload-to-temp-then-rename-into-place idiom used by many clients.
• Instant directory listings on buckets with millions of objects (optional metadata index).
• Free-space reporting (df, WinSCP's space display) backed by your quota.

Known limitations

Editing a file in place is not supported

Objects are replaced as a whole, never edited in place. Mounting the connection (sshfs, rclone mount) and opening a remote file in an editor, or appending to an existing file, will not work, these return a clean “operation unsupported” error rather than corrupting data. Upload a new version of the file instead.

Timestamps and permissions are not preserved

Object storage has no POSIX permission bits or modification-time metadata, so chmod/chown and timestamp-preserving flags (scp -p, rsync --times/--perms) report success but have no effect. Files list as 0644 and directories as 0755. If your workflow depends on preserved mtimes (e.g. rsync change-detection by time), use size/checksum-based comparison instead.

Rename is a server-side copy, not an atomic operation

Renaming an object copies it to the new key and deletes the old one. The familiar temp-then-rename upload pattern works, but strict atomic-replace guarantees aren't offered, and renaming a very large object takes as long as a copy. Renaming a directory is not supported, move the files individually (you'll get a clear message if you try).

Changes made outside Firepipe may take a moment to appear

If you write to your bucket directly (outside Firepipe) while using indexed listings, the new objects may take a short while to show up in directory listings as the index catches up. Listings served live from the bucket reflect changes immediately.

Very large single-directory listings are bounded

A single directory listing is capped per request and fails loudly if a folder is extraordinarily large, rather than silently returning a partial list. Spread very large object counts across sub-folders, or enable the metadata index.

SSE-KMS encrypted buckets are not yet supported

Buckets that enforce server-side encryption with AWS KMS (SSE-KMS), or whose policy denies unencrypted uploads, are not yet supported, use SSE-S3 (AES-256) instead. Our onboarding check will tell you at connection time if your bucket requires this, rather than letting it fail on your first upload. SSE-KMS support is on the roadmap.

Recommended: auto-clean interrupted uploads

Large files upload in multipart chunks. Firepipe aborts the upload cleanly on an ordinary error, but if a gateway node is hard-killed mid-upload (an out-of-memory kill or an instance loss), the already-uploaded parts can be left behind in your bucket, and your cloud provider bills for that orphaned storage until it's removed. This is rare, but it's your bill, so we'd rather flag it.

Add a one-line lifecycle rule to your bucket to clean them up automatically, e.g. on S3, “abort incomplete multipart uploads after 7 days” (AbortIncompleteMultipartUpload). It's a standard, safe setting that affects only unfinished uploads, never your real objects. We recommend rather than set this for you: Firepipe is never granted permission to change your bucket's configuration.

Questions?

If you're automating against Firepipe and need a specific behaviour confirmed, email [email protected], we'd rather scope it with you before you build than have it surprise you in production.