Does anybody here self-host a mail-by-proxy solution? If so, I’m interested to hear about your setup, experiences and any drawbacks. I have a custom domain and a hosted email service with a very small amount of storage. I’d like to host something locally so that I can keep all my email without stressing about the space. I also want to be able to use email on my phone and computer and a web interface for tablets or while traveling. Finally, I’d like emails that I send to be stored locally so I can search it. Does anybody else already do something like this? I can forge my own path, but oftentimes, somebody else is already doing it better.
You must log in or # to comment.
You’re asking for a lot of pain. That’s all I can say. Like SIP, SMTP is one of the most attacked services out there. It has to be public, it has to be on specific ports, and it has to be advertised that it’s available. There’s a reason why people don’t mess with it anymore.
I’d like to hide behind the service that I’m paying for without incurring extra fees for retaining it all. I can figure out the pull side by using fetchmail or something to a server that hosts dovecot, but the sending side is confusing since I’d need something that can receive my email and send it via the service. It’s only 1 email address, so I’m not looking for a mail relay, but something like a full caching mail proxy.
Just configured your mail clients pop/IMAP server as your fetchmail target and SMTP as your hosted service.
Just use Proton or a similar service. You’re getting the same thing for free or cheap.
Its possible to tell your mta (like postfix) to use another mta for all mails, or only some domains etc, so using a third party to play the internet facing service then getting the mails by fetchmail, storing them in a dovecot server is easy. on the sending part you could use your standard email client (i.e. thunderbird on pc or k9-mail on smartphone) to send it to your postfix instance that also sits on the server hosting your dovecot service. the mta there takes the mail and delivers it by rules which could just be using the mta of your freemailer using username/password of your account for all outgoing emails. i am doing this but the “external” mail system are my servers as well, i just don’t want emails to stay too long on VMs in the datacenter where i have no access to the physical disks in case something goes wrong.
a raspberry pi is sufficient for such a aetup (i am using a pi4 currently but for emails only i’ld say a 3 or older would do too), adding a disk via usb makes storage huge and cheap then, i use two usb ssd’s in a raid1 for storage… that server could be only accessible through vpn if you whish, depending on your skills and needs (i mainly use ssl client certificates that are supported by k9mail and thunderbird so it fits seamless to be connected through a haproxy that authenticates these before proxying the plain connection to the pi) clients like thunderbird can offline-store all emails (configure download-or-not per imap folder) making searches easy and quick while my k9 client can search locally or on the server if needed.
maybe adjust maximum mail size of your own mta to exactly match (or slightly less) that of the freemailer you use to prevent surprises of big but later then unsent emails.
its possible to have a nextcloud instance on that same pi that acts as an email web mailer just in case of (i really dont need it, but i’ve set this up anyway). nextcloud is also great for syncing/backup files pictures, contacts notes todo lists and calendar of your phone (where i use davx5 opentasks and foldersync for). there are other webmailers available but installing /using nextcloud is not a too bad idea either ;-)
i suggest also setting up some automatic offsite backup with snapshots of that pi then to cover emails and the setup and its configs ;-)
I think this is exactly what I’m looking to do. Thanks for such a detailed writeup!
I did some reading last night and think it lines up with what you’re saying. I found docker-mailserver with some configuration. The only thing I need to add is mail filtering to folders and I think that’s included.
you’re welcome.
what i’ld suggest… a general rule that i like to always follow is to use a test system for everything new. but that does not need to be a full separate system every time.
lets say you have your mailbox and want to try getting new mails from it using fetchmail. first you can use uidl mechanisms to only fefch every mail once and besides that leave them all on the server, but i like it a bit more secure: create a second email adress/account at your mail providers service only for testing. thus you can do whatever you like to to test the mechanisms only without even touching your real inbox (maybe even fill it up with large emails and look how the system reacts, i once had an email account with a cheap provider that deadlocked the inboxes when full…). then when everything is as you want it, switch the account and password (or create another config file for fetchmail) and your’re done. every change (not only fetchmail things) could go tested this way before going live with the changes. filtering could be done with procmail for example, but when the mda that is called by procmail somehow exits with success when the email really isn’t delivered, then the email might get lost forever depending on the settings of course. so fiddling with new stuff always carries the risk of not fiddling correctly ;-)
have fun !
Few weeks late to pitch in now but I can +1 docker-mailserver.
It has almost everything included and the configuration files are quite straightforward and flexible enough that you can drop little edits into the individual services if you need to tweak something.
My setup is very close to what you want: I use fetchmail to pull in from my old gmail and yahoo inboxes, I also have my own domain so I configured the MX records so that emails go straight to my server, with a fallback to my email provider (any mail that doesn’t make it directly to my mailserver will still get pulled into my inbox with fetchmail when it comes back online).
Docker-mailserver allows you to set the SMTP of your instance to use your provider. This is important because it means that they do all the reputation stuff so that your emails work properly (and both my home ISP and my VPS provider don’t do sending over pot 25 anyway).
So when I need to connect a new client (like Thunderbird) to my email I don’t need to manually config anything as docker-mailserver has all the auto config messages so its really seamless. At the same time my risk is low because even if my sever is off my provider will still receive anything on my behalf. I can only send using the username I have paid for from my provider and switching between gmail and yahoo is not possible without rewriting configs and restarting services but its not something I want anyway. On the receiving side I can have any number of aliased usernames that will all be received by my server (but only when its on so i use them rarely and for disposable addresses).
Big downsides are:
- backups are now my problem as I don’t keep duplicates.
- I route my traffic via a VPS+VPN to get a static public IP address - this was a headache to get all the little details just right but its stable now
- I have to host separately my own webmail, although I’m mostly using my phone with k-9 Mail and Thunderbird (I use roundcube)
- Getting server side filtering rules working was also annoying and so far I still have to add new rules through roundcube (there was a plugin for Thunderbird but I don’t want to open the additional ports required)
- !!! Spam !!! Docker-mailserver has a great Rspamd default settings out the box so its actually fine but now I have to manage all the additional rules and its not super intuitive especially because I am doing all this just for myself (yay!) but the tools are clearly meant for managing a fleet of inboxes so everything takes me longer to figure out
- integrating contacts is not included and might be important for your experience (again I was able to add this as a plugin to roundcube … eventually)
Most of my complaints stem from the fact that I’m not very good at this but in the end it has been very satisfying to drop the occasional: “I host my own email BTW”
Good luck! Let us know how you get along!
I just found out my emails were being marked as spam just using a custom domain through icloud mail. So I don’t envy those having to run and troubleshoot their own hosted system.
Was this due to DMARC/DKIM, SPF or something else?
No clue man, sorry
I run my own email server using mailcow-dockerized. Ironically, the problem is not enough volume.
Email archiving? You can probably find self-hosted products for that. Or any service that speaks a compatible protocol with your host can fetch mail from the host and delete it on the remote side to save space.
