Unlimited history for bash shell


misc bash shell command-line

This post is intended as a guide to increase the size of bash history (or even better, configure it to have an unlimited size). We will try to understand the different environment variables involved and tweak them to have a bigger(or unlimited) history.

Why do we need more history?

As developers, we find ourselves trying to remember a certain command we used in the past (which is pretty lengthy to type or uses a specific combination of options/arguments). The reverse history search (ctrl-r) feature comes in very handy in such situations. This command lets us search through the history of commands stored in the history file.

By default, the number of commands persisted in the history is just 500 - This is insufficient, compared to the number of commands we type on a typical day. Once this limit is hit, older commands get truncated as newer commands are added to the history file. I verified that I type around 100 commands a day, which means I’ll only retain 5-days worth of commands in my history and lose everything before that. By increasing the size of our history, more commands can be retained and be available for searching. This way, we can search for commands used months(or years) ago.

Before we get started, let’s try to understand a few environment variables pertaining to history in bash.

HISTFILESIZE - Refers to the maximum number of commands to store in the bash history file (Default is 500)

HISTSIZE - Refers to the maximum number of commands to load into memory from the history file (Default is 500)

HISTFILE - Refers to the path/location of the history file (Default is ~/.bash_history)

These are the variables we will be tweaking for a bigger history. You can find more details about these variables from the man page for bash (Try man bash from the commandline).

Steps to setup bash to have unlimited history

1. Change the bash history file

As documented in the StackOverflow post, there could be issues with the default bash history file getting truncated in certain cases. We can workaround this problem by modifying the HISTFILE env variable to use a different file for storing history.

# In ~/.bashrc
export HISTFILE=~/.history

2. Increase history size

Setting the HISTFILESIZE and HISTSIZE variables to an empty string makes the bash history size unlimited.

# In ~/.bashrc
export HISTFILESIZE=
export HISTSIZE=

In bash 4.3 or later, you can set these to -1 as well, to have the safe effect. (Rather than setting it to UNLIMITED ), some set this to a really high value (100000), so that the history file doesn’t grow indefinitely.

3. Copy existing history

Though we start using a different history file, we shouldn’t lose commands already on the old history file. Let’s just copy the contents of the old file to new one.

# On the command line
cp ~/.bash_history ~/.history

Some nice-to-haves

Now that we have increased the size of our history, let’s try to make it more robust (And that means meddling with more environment variables!).

1. Immediate append

PROMPT_COMMAND="history -a; $PROMPT_COMMAND"

PROMPT_COMMAND - Setting this to history -a flushes the command to the history file immediately (otherwise, this would happen only when the shell exits, and you could lose history upon unexpected/unclean termination of the shell).

2. Add Timestamp to history

# In ~/.bashrc
export HISTTIMEFORMAT="[%F %T] "

HISTTIMEFORMAT - Setting this ensures timestamps are written to the history file (along with the actual command ). You can later find out the exact time of execution for a command, if needed. Also, using the history command to view the last n commands displays the timestamp information, in addition to the actual command:

# On the command line
~$ history 5
 9626  [2019-12-03 21:51:02] cd ~
 9627  [2019-12-03 21:51:04] ls -lrt
 9628  [2019-12-03 21:51:08] df -h
 9629  [2019-12-03 21:51:12] man bash
 9630  [2019-12-03 21:51:16] history 5

3. Handling duplicate commands

(While searching with Ctrl+R) Stepping through the history with UP and DOWN keys becomes a bit annoying if the same command comes up again and again. A better option is to skip duplicates and show each command only once with the following setting. (Duplicate commands are still written to the history - However, they come up just once while stepping through).

# In ~/.bashrc
export HISTCONTROL=ignoredups

We can take it up a notch by not writing duplicates to the history file at all, using the erasedups option.

 # In ~/.bashrc
 export HISTCONTROL=erasedups

The ensures all previous lines matching the current command is removed from history before the current command is saved.

4. Sharing history

When we have multiple bash shells open simultaneously, a command entered in one shell can be made available to all other shells with the following:

PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"

As we know already, history -a appends the command to history file immediately. Additionally, history -c clears the in-memory history and history -r appends the contents history file to the in-memory history. (Together , history -c + history -r gives the effect of refreshing the in-memory history with the updated history file, after every command). I personally don’t have this enabled since there is not much value add, and refreshing history after every command can get expensive as our history file grows in size.

Putting everything together

# In ~/.bashrc
export HISTFILESIZE=
export HISTSIZE=
export HISTFILE=~/.history

export HISTTIMEFORMAT="[%F %T] "
PROMPT_COMMAND="history -a; $PROMPT_COMMAND"
export HISTCONTROL=ignoredups
# On the command line
cp ~/.bash_history ~/.history

Conclusion

In this post, we saw how to make our bash history bigger, so that searching for past commands is seamless. Personally, I didn’t think there was so much to bash history(Boy, was I wrong!), and learnt a lot of things while setting this up - I hope you found the post useful too. See you in the next post.

See Also