When comparing VLC vs HandBrake, the Slant community recommends HandBrake for most people. In the question“What are the best combined audio & video transcoders?” HandBrake is ranked 2nd while VLC is ranked 4th. The most important reason people chose HandBrake is:
There are presets for everything, so you don't have to delve deep in to advanced features if you don't want or need to. And in most cases you won't have to because the presets are great.
Specs
Ranked in these QuestionsQuestion Ranking
Pros
Pro Free, open source, and cross-platform
VLC is licensed under the CC-SA v3.0+ and available on Windows, OSX, and Linux with source code available here.

Pro Simple to setup and use
VLC is an incredibly robust application but very simple on the surface. It makes playing music simple, yet still manages to give the user all the tools he/she wants in a music player.
Pro Very versatile and easy to use
Pro Doesn't require additional codecs
Everything VLC needs to play media files is contained within which means no outside codecs are needed. This makes it one of the most hassle-free music players as it can play virtually anything as soon as it's installed.
Pro Tons of advanced settings
Besides basic configurations, video player has an extensive amount of adjustable settings.
Pro Doesn't manage your music library for you
You organize your music into folders any way you want. Want to play an album? Drag and drop. No tags, no confusion, no fluff. Easy.
Pro Can be almost completely keyboard-driven
VLC includes keyboard shortcuts for most actions. The video player can be controlled with simple and customizable keyboard commands.
Pro Surprisingly Lightest on Resources
Yes. Lightest. Even better than the ones known to be light. Both when playing music, only a single song or when playing a video. Either in terms of Ram and Cpu. Compared with almost all, including the ones from Windows like mpc-hc .. or light ones from Linux like Alsa Player, Audacious, SmPlayer .
Pro Can play every format video available (as of 2017)
Pro Supports a GUI interface and an ncurses terminal interface
Pro Can easily stream music across your home network from any device on the network
Using sftp/ssh music (and videos) can be streamed from any server on the network to any device running VLC. VLC can also be used as a webcam for streaming and snapshots. Amazing all in one package
Pro Large amount of extensions
VLC has a large amount of extensions.
Pro Great, easy to use presets
There are presets for everything, so you don't have to delve deep in to advanced features if you don't want or need to. And in most cases you won't have to because the presets are great.
Pro Advanced features
Although HandBrake is pretty straightforward to use, you can adjust pretty much any aspect of your conversion if necessary. For example, when transcoding video you have the option of adjusting between constant and variable framerate, adjusting average bitrate and constant quality, having 2-pass encoding or not, as well as tinkering with encoder specific options, and many, many, many more options.
Pro Fast
Since version 0.9.9. HandBrake has been very fast.
Pro Wide range of formats and multiple media types supported
HandBrake can handle DVDs, Blu-Rays, .mp4, .mkv, H.264(x264), MPEG-4, MPEG-2, AAC, MP3, FLAC, AC3, Vorbis, AC-3, DTS and DTS-HD among others.
Pro Often works when dumping archiving disc to hard drive methods do not
Pro Free and open source
HandBrake is licensed under GPL.
Pro Cross-platform
HandBrake works on Windows, Linux and OS X.
Pro Lightweight
It is designed to convert, no added bloat.
Pro Good metadata support
HandBrake can pull, use and edit metadata.
Pro Command Line Interface option
There's HandBrakeCLI if you wish to use HandBrake frome a terminal.
Pro Encoding options are comprehensive but easy to use up front
The GUI makes it easy to encode by providing profiles and a simple GUI, but offers extensive encoding options for people willing to learn and spend time experimenting.
Cons

Con Not really meant to be a music player
VLC is made to be a video player. Therefore, it won't carry out actions such as managing your music library.
Con Lacks libraries and advanced music player features
VLC is a media player first and foremost. There is no library management (aside from playlists), limited usage of tags, and no rating system. VLC is best at playing a file directly from a folder, but falls behind when it comes to helping you manage or find good songs in your music library.
Con Absence of personalization
VLC Player has only one basic interface in white or black, and overall personalization opportunities are quite narrow. Unfortunately, you're pretty much stuck to the default look.
Con Contains some insignificant but irritating flaws
Despite all keyboard shortcuts, the width of the picture can't be controlled by trackpad. Besides that, the VLC start-up window doesn’t open at the same place or the same width at which it was previously closed.
Con Supports only two containers
You can output only .mp4 and .mkv.
Con There is no way to preserve menus and special features
Menus and special features will typically not be included in the output from a handbrake encode. Third party software would need to be used.
Con no linux support for hw acceleration
I know that it is not as high quality, but transcoding terabytes of 1080p videos to h265 without hw support isn't realistic and wont be for a long time
Con Not for 1:1 archiving or true backup use
Most uses of Handbrake are lossy, lossless is possible but it usually entails crazy huge file sizes.
