[VM] vm-save-buffer

[Uday Reddy]

There are two functions called `vm-save-buffer' and `vm-save-folder'.

- `vm-save-folder' is the correct one to use. It does more.
- But `vm-save-buffer' is bound to C-x C-s.

Does anybody `vm-save-buffer' instead of `vm-save-folder'? For what purpose?

There is a proposal to get rid of it (or at least the key binding).

Cheers,
Uday

[Gian Uberto Lauri]

I have ti check, but I think I'll change the function used.

--
Gian Uberto

Post by Uday Reddy
There are two functions called `vm-save-buffer' and `vm-save-folder'.
- `vm-save-folder' is the correct one to use. It does more.
- But `vm-save-buffer' is bound to C-x C-s.
Does anybody `vm-save-buffer' instead of `vm-save-folder'? For what purpose?
There is a proposal to get rid of it (or at least the key binding).
Cheers,
Uday

[Gian Uberto Lauri]

Post by Gian Uberto Lauri
I have ti check, but I think I'll change the function used.

NOt using vm-save-buffer.
--
ing. Gian Uberto Lauri
Senior Solution Developer
Dir. Tecnica Innovazione Ricerca / Tech. Innovation & Research Div.
***@eng.it

Sun Java Certified Programmer

Engineering Ingegneria Informatica spa
Corso Stati Uniti 23/C, 35127 Padova (PD)

Tel. [phone] | main(){printf(&unix["\021%six\012\0"],
Fax [phone] | (unix)["have"]+"fun"-0x60);}
http://www.eng.it | David Korn, AT&T Bell Labs
| ioccc best One Liner, 1987

[John Stoffel]

Uday> There are two functions called `vm-save-buffer' and `vm-save-folder'.
Uday> - `vm-save-folder' is the correct one to use. It does more.
Uday> - But `vm-save-buffer' is bound to C-x C-s.

Uday> Does anybody `vm-save-buffer' instead of `vm-save-folder'? For what purpose?

Uday> There is a proposal to get rid of it (or at least the key binding).

Get rid of it completely... and re-bind C-x C-s to vm-save-folder
instead. It's not called from anywhere else inside VM, so just remove
it and release VM-8.2.0c...

I'm in the middle of starting to deploy dovecot and IMAP for all my
mails, just so I can read them on my phone as well as inside VM. But
I'm starting to think I just need to move to mutt and rebind most of
it's key bindings to reflect my VM experience.

I'd hate to do it, but since I have negative elisp coding skills, and
since the project is effectively dead now, I'm not sure what else can
be done...

I do want to thank Uday for all his work over the years, I do
appreciate this thankless task he's taken on here.

John

[Johan Vromans]

On Wed, 4 Nov 2015 16:17:24 -0500

Post by John Stoffel
I'm in the middle of starting to deploy dovecot and IMAP for all my
mails, just so I can read them on my phone as well as inside VM. But
I'm starting to think I just need to move to mutt and rebind most of
it's key bindings to reflect my VM experience.
I'd hate to do it, ...

Same here. On the Linux desktop I switched to claws-mail and I must
say that the switch caused less problems than I thought. Since claws-mail
is sufficiently different from Emacs/VM there's little confusion in key
bindings and such. Muscle memory doesn't get in the way :) .
I use emacs (via emacsclient) as external editor when needed.

I still use VM to maintain my collection of legacy mbox files.

-- Johan

[Daniel Barrett]

Post by John Stoffel
I'm starting to think I just need to move to mutt and rebind most of
it's key bindings to reflect my VM experience.

Same here. On the Linux desktop I switched to claws-mail...
I still use VM to maintain my collection of legacy mbox files.

I've been using VM for nearly 25 years and maintain over 3500 mail
folders with it. I have yet to see another email program that scales
this well and is this flexible. If VM lacks a feature (example: a
trash can), I just make one myself in elisp. My only criticisms are
tiny compared to the immense value that VM has brought me over the
decades.

Stick a "Donate" button on the VM Launchpad page and I'll gladly
contribute.

--
Dan Barrett
***@blazemonger.com

[Manuel Hermenegildo]

Post by Daniel Barrett
I've been using VM for nearly 25 years and maintain over 3500 mail
folders with it. I have yet to see another email program that
scales this well and is this flexible.

Totally agreed. Huge kudos to Uday. VM rules! Cheers, --Manuel

[Johan Vromans]

On Wed, 4 Nov 2015 20:35:40 -0500

Post by Daniel Barrett
I've been using VM for nearly 25 years and maintain over 3500 mail
folders with it. I have yet to see another email program that scales
this well and is this flexible.

Same here, although I have only 1000 mail folders (mbox files).
I think I started with Emacs 16 or 17...

However, when a mail folder gets big (more than 2000 messages or so)
VM really slows down. With big IMAP folders it becomes painfully slow
when saving (due to the local cache of 100+Mb) and autosave (or poll?)
makes Emacs unresponsive for several seconds every couple of minutes.

This was the time I decided I really needed to start looking for an
alternative mail client.

Post by Daniel Barrett
If VM lacks a feature (example: a trash can), I just make one myself
in elisp.

I've made many extensions to my VM (and Emacs) environment and you
know what? Sometimes Emacs API changes, VM API changes, BBDB API
changes and the extensions break. I found it's no fun trying to mend a
big lisp function that I wrote 15 years ago and haven't touched ever
since.

So all advantages have their drawbacks as well. Which one outweights
the other is very individual.

Post by Daniel Barrett
My only criticisms are tiny compared to the immense value that VM
has brought me over the decades.

Don't get me wrong. I'm still a big fan of Emacs and I'm very happy
with what VM has given me all those years. But I got to the point that
I needed something else for my mail.

(And yes, I did try WanderLust first.)

-- Johan

[Daniel Barrett]

...when a mail folder gets big (more than 2000 messages or so)
VM really slows down.

Hmm, I have a 2500 message Amazon folder (mbox format) that opens in
half a second. That might be because I have a fast 12-core CPU and an
SSD, but just in case, here is something for you to try:

1. Open your mbox file in Emacs like a normal file. (Not with VM.)
2. Globally replace all mail header lines that begin with "X-VM-",
changing them to begin with "XX-VM-". This will make VM ignore these headers
the next time you visit this folder. Save and close the file.
3. Visit this mbox in VM. VM will recalculate all the headers you
just renamed, which might take some extra time.
5. Save the folder and close it.

Now open your folder again. Is it faster? VM has some kind of bug
where these headers get out of whack, and deleting/recreating them
improves performance again.

With big IMAP folders it becomes painfully slow when saving (due to
the local cache of 100+Mb) and autosave (or poll?) makes Emacs
unresponsive for several seconds every couple of minutes.

Interesting. I don't use IMAP so I've never noticed this. (I use
fetchmail to retrieve my email from the IMAP server and then
process it all locally.)

So all advantages have their drawbacks as well. Which one outweights
the other is very individual.

Very true!

--
Dan Barrett
***@blazemonger.com

[Johan Vromans]

On Thu, 5 Nov 2015 10:41:21 -0500

Post by Daniel Barrett

...when a mail folder gets big (more than 2000 messages or so)
VM really slows down.

Hmm, I have a 2500 message Amazon folder (mbox format) that opens in
half a second. That might be because I have a fast 12-core CPU and an
SSD,

On my 2-core 6GB CPU w/ SSD, the IMAP cache *loads* within a second, but
takes between 15 and 20 seconds to save.

-- Johan

[Daniel Barrett]

Post by Johan Vromans

Post by Daniel Barrett
Hmm, I have a 2500 message Amazon folder (mbox format) that opens in
half a second. That might be because I have a fast 12-core CPU and an
SSD,

On my 2-core 6GB CPU w/ SSD, the IMAP cache *loads* within a second, but
takes between 15 and 20 seconds to save.

Ah, we are comparing apples and oranges then. I'm not reading over
IMAP. I'm reading and saving mbox files directly on local disk.

--
Dan Barrett
***@blazemonger.com

[Johan Vromans]

On Thu, 5 Nov 2015 15:20:51 -0500

Post by Daniel Barrett
Ah, we are comparing apples and oranges then. I'm not reading over
IMAP. I'm reading and saving mbox files directly on local disk.

The IMAP cache file *is* an mbox file on local disk.

-- Johan

[John Stoffel]

As I really understand it, isn't part of the problem that Emacs
buffers are limited to something like 256Mb, so we're also hitting
some of those limits with big folders?

And I don't have as big a system as that, and for me the key advante
of VM is that I can edit in my favorite editor, and it's fast. It
works well over SSH and inside screen.

It's just now thatI really want to setup IMAPS for my email (I run my
own domain, so this is both learning and paranoia calling me!) so I
can see the darn images inside emails I get from time to time.

I've got over 2500 mail folders, some with over 2000 messages from the
past 20+ years. VM is plenty fast, easy to search with (though I
admit I do NOT do virtual folders at all...)

So I'm not really excited about leaving VM, but IMAP support is
becoming more and more important to me, and so far, Mutt does the best
job. It's fast, powerful and works well inside SSH.

And I really feel that development has stalled. There hasn't been a
release in years now. I've asked Uday to just post whatever he has,
but that hasn't happened. I understand his desire to not put out
buggy code, but maybe that's what needs to happen.

So right now I'm slowly, oh so slowly... writing the .muttrc file with
the VM keybindings that I need so as to make my transition as painless
as possible. It will never be perfect, but good enough will be fine I
think.

John

[Yeechang Lee]

Post by John Stoffel
I've got over 2500 mail folders, some with over 2000 messages from
the past 20+ years. VM is plenty fast, easy to search with (though
I admit I do NOT do virtual folders at all...)

I, too, have been using VM for more than 20 years, and only began
using virtual folders this year. It's the best way to search.

V C [field you want to search] [text to search for]

shows you all messages that match in a virtual
folder. vm-virtual-mirror is, I believe, set by default to t so
changes made in the virtual folder will be reflected in reality.

[j***@laposte.net]

Post by John Stoffel
I've got over 2500 mail folders, some with over 2000 messages from the
past 20+ years. VM is plenty fast, easy to search with (though I
admit I do NOT do virtual folders at all...)

Same with me. More than 20 years. Over 16 Mo in VM.

Post by John Stoffel
So I'm not really excited about leaving VM, but IMAP support is
becoming more and more important to me, and so far, Mutt does the best
job. It's fast, powerful and works well inside SSH.

I love VM too.

Post by John Stoffel
And I really feel that development has stalled. There hasn't been a
release in years now.
So right now I'm slowly, oh so slowly... writing the .muttrc file with
the VM keybindings that I need so as to make my transition as painless
as possible. It will never be perfect, but good enough will be fine I
think.

Has anyone looked at Mu4e?

--
JL

[Hugo Geir]

Hello,

Post by j***@laposte.net
Has anyone looked at Mu4e?

I'm using it currently.
Searches are excellent.

What I miss the most is dealing easily with signing/encrypting
messages.

I have ceased using vm many years ago, unfortunately. I have
used Thunderbird until recently, when searching broke down for
me completely. I could, however, copy all messages to a dovecot
based IMAP mailbox. So I'm uncertain whether to stick with mu4e.
I do have notmuch and mutt on the list to try out. Well, maybe I
should do a serious attempt to configure VM to deal with a few
big IMAP folders (approx 20k messages).


A big Thank You! from my side as well.

Cheers,
Erich

[Uday Reddy]

Post by Johan Vromans
However, when a mail folder gets big (more than 2000 messages or so)
VM really slows down. With big IMAP folders it becomes painfully slow
when saving (due to the local cache of 100+Mb) and autosave (or poll?)
makes Emacs unresponsive for several seconds every couple of minutes.

I don't think the number of messages matters much, but the total size of the
folder make a difference because the entire folder is stored in the main
memory.

You can cut down the size of the folder by using "external messages"
(Section 1.4 of the VM Manual). I let my folders go up to 5000-6000
messages without any problem.

The autosave behaviour may vary between different Emacs versions. On Emacs
24.2, I don't get any pauses at all. My settings are:

(setq large-file-warning-threshold 100000000)
(setq auto-save-timeout 900) ; default is 30
(setq auto-save-interval 1000) ; up from the default of 300

Cheers,
Uday

[Yeechang Lee]

Post by Uday Reddy
You can cut down the size of the folder by using "external messages"
(Section 1.4 of the VM Manual). I let my folders go up to 5000-6000
messages without any problem.

What is this feature? I can't find it in section 1.4 or elsewhere in
the manual accompanying 8.1.2.

[Uday Reddy]

Post by Yeechang Lee

Post by Uday Reddy
You can cut down the size of the folder by using "external messages"
(Section 1.4 of the VM Manual). I let my folders go up to 5000-6000
messages without any problem.

What is this feature? I can't find it in section 1.4 or elsewhere in
the manual accompanying 8.1.2.

You will find it 8.2.0a and 8.2.0b prereleases:

https://launchpad.net/vm

Cheers,
Uday

[Johan Vromans]

On Thu, 5 Nov 2015 18:09:36 +0000

Post by Uday Reddy
https://launchpad.net/vm

Okay. Upgraded to 8.2.0b.

IMAP cache load time is still within a second. The cache is 130MB and
it takes approx. 15 seconds to save. This is roughly the same as with
VM version 8.1.2.

With external messages, the cache is only 15MB and saves acceptably fast.

Good.

BUT...

*** Initial load of the IMAP cache often stops after a number of messages:

Retrieving message 346 (of 1689) from Home:Inbox, 100%... [3 times]
Retrieval from Home:Inbox signaled: (vm-imap-protocol-error expected
(BODY[] string) in FETCH response) Retrieving message 346 (of 1689) from
Home:Inbox, 100%... Updating summary...
Parsing messages... done
Expunging messages in cache... done
348 messages, 0 new, 0 unread, 0 deleted

Hitting 'g' a number of times makes the loading complete.

*** I somehow lost my colours in the summary buffer.

*** Very annoying (and dangerous!) change in behavour:

In the summary window:
Assume message 1688 is selected (blue).
Click on message 1681, or move cursor to this message.

Type 'D' (delete). Message 1688 is marked for deletion.

In VM version 8.1.2, message 1681 would become selected and marked for
deletion.

The same thing happens with U (undo), MM etc.
Note that e.g. o (save) *does* operate on the message the cursor is on.

*** VM 8.2.0b is a prerelease and dates from Dec 27, 2011. It's been
unmodified for almost 4 years. So it is either rock stable (and should have
been officially released long ago), or it is dead.
From the manual: "Version 8.2.0, planned for release in August/September,
2010." Really?

Sorry, but this does not help me regain confidence in VM.

-- Johan

[Uday Reddy]

Post by Johan Vromans
Retrieving message 346 (of 1689) from Home:Inbox, 100%... [3 times]
Retrieval from Home:Inbox signaled: (vm-imap-protocol-error expected
(BODY[] string) in FETCH response) Retrieving message 346 (of 1689) from
Home:Inbox, 100%... Updating summary...
Parsing messages... done
Expunging messages in cache... done
348 messages, 0 new, 0 unread, 0 deleted
Hitting 'g' a number of times makes the loading complete.

That looks like your IMAP server was disconnecting.

Post by Johan Vromans
*** I somehow lost my colours in the summary buffer.
Assume message 1688 is selected (blue).
Click on message 1681, or move cursor to this message.
Type 'D' (delete). Message 1688 is marked for deletion.
In VM version 8.1.2, message 1681 would become selected and marked for
deletion.

You might need to get rid of old cruft in your .vm file. In particular, if
you were using `u-vm-color' (a non-standard add-on), you need to get rid of
it and use VM's summary faces. See the NEWS file and make sure that all the
CHANGES are incorporated in your .vm file.

Post by Johan Vromans
*** VM 8.2.0b is a prerelease and dates from Dec 27, 2011. It's been
unmodified for almost 4 years. So it is either rock stable (and should have
been officially released long ago), or it is dead.
From the manual: "Version 8.2.0, planned for release in August/September,
2010." Really?
Sorry, but this does not help me regain confidence in VM.

I am only one person maintaining VM, and that limitation is not going to
disappear any time soon. I myself didn't have "confidence" in VM after Kyle
Jones retired. I regained it only after I ensured that I could fix things
myself if things went wrong. So I can understand if people feel nervous
about sticking to VM.

In retrospect, I also see that I have been overly cautious about making
public releases. It has been mostly due to the way Linux distributions
function, allowing users to upgrade to a new version with the click of a
button. But all my prereleases have been perfectly stable (except for one,
which I had to re-release a week later). In retrospect, all the prereleases
could have been public releases.

So, in future, you can expect more frequent public releases. Whether it will
be good or bad is another question.

Cheers,
Uday

[John Stoffel]

Uday> I am only one person maintaining VM, and that limitation is not
Uday> going to disappear any time soon. I myself didn't have
Uday> "confidence" in VM after Kyle Jones retired. I regained it only
Uday> after I ensured that I could fix things myself if things went
Uday> wrong. So I can understand if people feel nervous about sticking
Uday> to VM.

I wish I could do more to program in elisp, but it's just not going to
happen. Sorry.

Uday> In retrospect, I also see that I have been overly cautious about
Uday> making public releases. It has been mostly due to the way Linux
Uday> distributions function, allowing users to upgrade to a new
Uday> version with the click of a button. But all my prereleases have
Uday> been perfectly stable (except for one, which I had to re-release
Uday> a week later). In retrospect, all the prereleases could have
Uday> been public releases.

I think if you just setup a git repository with a beta branch, and
some stable branches, you'd be just fine. Release lots of beta
attempts, but only a few release versions and things will be fine.

Uday> So, in future, you can expect more frequent public
Uday> releases. Whether it will be good or bad is another question.

Breaking things is ok, honestly! I also hope that more frequent
updates, and maybe even more ability for people to clone and branch VM
will make development pickup again.

Who knows, maybe even I'll take up the mantle and learn (e)lisp enough
to program in it. Stranger things have happened. *grin*

And please know I do apprecaite all you've done Uday!

John

[Uday Reddy]

Post by John Stoffel
I think if you just setup a git repository with a beta branch, and
some stable branches, you'd be just fine. Release lots of beta
attempts, but only a few release versions and things will be fine.

VM is currently on a bzr repository on Launchpad. The README file gives
instructions for getting the development versions.

Cheers,
Uday

[Johan Vromans (by way of Johan Vromans )]

Uday,

Please do not get me wrong -- I really, really much appreciate the work
you've done on VM. And, as I wrote in my initial mail, I really hate to
have to replace VM by anything else. But I'm just running into the situation
that VM seems no longer capable of doing what it is supposed to do.

On Thu, 5 Nov 2015 20:35:36 +0000

Post by Uday Reddy
That looks like your IMAP server was disconnecting.

Possibly. But I've never seen this with 8.1.2, while it consistently happens
with 8.2.0. Fortunately, only on the initial load of the IMAP mailbox.

BTW, when I delete the cache and re-visit the IMAP mailbox in 8.1.2, a new
cache is created. When I do this in 8.2.0, it gets stuck on an (empty)
read-only Inbox since it thinks it must recover the auto save file.

Post by Uday Reddy

Post by Johan Vromans
*** I somehow lost my colours in the summary buffer.

You might need to get rid of old cruft in your .vm file. In particular,
if you were using `u-vm-color' (a non-standard add-on), you need to get
rid of it and use VM's summary faces. See the NEWS file and make sure
that all the CHANGES are incorporated in your .vm file.

I'm not using u-vm-color, which was introduced after stable 8.1.2.

When I use M-x vm-summary-faces-mode, it bails out with

apply: Marker points into wrong buffer: #<marker at 80 in Inbox>

Post by Uday Reddy

Post by Johan Vromans
Assume message 1688 is selected (blue).
Click on message 1681, or move cursor to this message.
Type 'D' (delete). Message 1688 is marked for deletion.

And 1689 becomes selected. Now if you were looking at message 100 or so,
everything happening to 1688 etc would be out of sight. Hey, I just did d,
didn't I? Apparently not. Let's hit d once more. Nope, still no effect...

Just in case the impact of this is not clear: It causes you to silently
lose messages since you're deleting the wrong ones.

Post by Uday Reddy

Post by Johan Vromans
In VM version 8.1.2, message 1681 would become selected and marked for
deletion.

I couldn't find anything in the NEWS/CHANGES that would have warned me for
such a significant change.

Post by Uday Reddy
So, in future, you can expect more frequent public releases. Whether it
will be good or bad is another question.

I think it is good.

[Ulrich Mueller]

Post by Johan Vromans (by way of Johan Vromans )
On Thu, 5 Nov 2015 20:35:36 +0000

Post by Uday Reddy
So, in future, you can expect more frequent public releases.
Whether it will be good or bad is another question.

I think it is good.

Not sure if it has been brought up before, but I think that switching
the repository from bzr to a less exotic VCS like git might also help.
Especially since GNU Emacs itself has made that transition.

Unfortunately, I can't say that I have contributed to VM recently.
However, the repository being in bzr certainly hasn't encouraged me in
the past.

Ulrich

[Joe Malcolm]

Post by Uday Reddy

Post by Johan Vromans
Assume message 1688 is selected (blue).
Click on message 1681, or move cursor to this message.
Type 'D' (delete). Message 1688 is marked for deletion.
In VM version 8.1.2, message 1681 would become selected and marked for
deletion.

I have noticed this change as well. I don't like it either.

Post by Uday Reddy
You might need to get rid of old cruft in your .vm file. In particular, if
you were using `u-vm-color' (a non-standard add-on), you need to get rid of
it and use VM's summary faces. See the NEWS file and make sure that all the
CHANGES are incorporated in your .vm file.

I'm not using this addon, so that's not it for me. It might be worth
thinking about whether it should be in the vm package if it's not to
be used.

Joe

--