When comparing Xfce vs spectrwm, the Slant community recommends spectrwm for most people. In the question“What are the best window managers for Linux?” spectrwm is ranked 9th while Xfce is ranked 18th. The most important reason people chose spectrwm is:
Spectrwm behaves largely like Xmonad (which is a good thing) without the ~700 MB GHC dependency and with plain text config files
Specs
Ranked in these QuestionsQuestion Ranking
Pros
Pro Low system resource consumption
Not just helpful for older computers where few system resources are available, but also simply for those who want to get the most out of their systems.
Pro Highly customizable
Xfce offers plenty of settings, and even things like theming XFWM is a simple task (it's just a handful of images.)
Many possible permutations of window colors, borders, fonts, etc. Compositing can make it look downright sexy.
Pro Works on a wide variety of platforms
Xfce can be installed on several UNIX platforms. It is known to compile on Linux, NetBSD, FreeBSD, OpenBSD, Solaris, Cygwin and MacOS X, on x86, PPC, Sparc, and Alpha.

Pro Classic and familiar
Xfce works very much like the classic Gnome & Windows desktops, taskbars (panels) and desktop icons, letting you get your work done without being frustrated.
Xfce embodies the traditional UNIX philosophy of modularity and re-usability. It consists of a number of components that provide the full functionality one can expect of a modern desktop environment. They are packaged separately and you can pick among the available packages to create the optimal personal working environment.

Pro Rock solid stability
Xfce will never be the cause of your crash.
Pro Designed for productivity
It loads and executes applications fast, while conserving system resources.
Pro Adheres to standards
A priority of Xfce is adherence to standards, specifically those defined at freedesktop.org allowing for interoperability and shared technology for X Window System desktops. This interoperability is particularly significant for users looking to, e.g., run alternative window managers.
Pro A true UNIX Desktop Environment
Xfce adheres to the UNIX philosophy, which means it strives for being modular, minimal and expandable. This makes it very much customizable. You can make it as minimal as you want and as heavyweight as you want depending on the features and modules/plugins you use.
Pro Does what it's meant to do easily and efficiently
XFCE is a desktop environment first and foremost, it does not waste time being overly flashy or by being bloated with features.
Pro Best for newcomers
Any one new to Linux feels comfortable using it.

Pro Low resource usage combined with flexible configuration
Pro Window manager (XFWM) is a compositing WM by default
By having a compositing WM as the default WM makes way for a lot of visual tweaks and tricks that can and do make Xfce look great. You can adjust the transparency, shadows, borders, etc. and many other advanced tweaks are also available.
Pro Well defined Session Manager
Pro Easy to export or import configurations
Pro like Xmonad light
Spectrwm behaves largely like Xmonad (which is a good thing) without the ~700 MB GHC dependency and with plain text config files
Pro Has a plain-text config file that it can reload while it's running
The config file can be reloaded while the WM is running, allowing the user to see the results of editing the config without logging out and back in again.
Pro Sane defaults
Inspired by xmonad and dwm, spectrwm has defaults that any normal user would enjoy rather than using an odd language or asymmetric window layouts.
Pro Straight-forward
Contains a basic set of options and doesn't require a language to configure it.
Pro Great for beginners
The defaults, simple design, and plain text config file make spectrwm a fantastic WM for those who aren't that familiar with Haskell, for example, and who just wish to get something substantial up and running.
Pro Supports floating windows
Spectrwm offers built-in keyboard shortcut support for floating windows.
Cons
Con No HiDPI support
Since Xfce is still based on GTK2 there is no HiDPI support (scaling UI elements).
Con Looks dated
It just looks like a 20 year old desktop in its stock form. However, it is possible for you to to give it a more elegant look using themes, icons and other customizations.
Con Screen tearing issues
The built-in compositor for Xfce does not handle VSync, meaning that it does not address screen tearing for those with Intel integrated graphics. A third party solution will have to be used for those that do want VSync such as using Nvidia proprietary drivers to handle VSync or installing a third party compositor such as Compton.
Con Looks ugly out of the box
Out of the box, Xfce is the one of the ugliest if not the ugliest DE out there. It definitely can become the most beautiful and gorgeous DE after a bit of tinkering and theming, but the default theme is not that good.
Con Missing some basic functionality for a desktop environment
Xfce is missing essential functionality like a file-archiver or a polkit-client, so you have to find alternatives for those applications (eg: by stealing them from MATE or GNOME, however this adds additional dependencies that will bloat Xfce).
Con Lacks modern design and effects
No support for transparency, effects in opening or closing a file browser, or other effects like cube or cylinder, unlike, say, KDE.
Con Terrible project infrastructure
The whole project is split across various sites so contributing is really hard. You also need to register on every site separately.
Con Lack of useful tools
Con One pixel wide window borders
The non-configurable, one pixel wide window borders make resizing difficult. Work-arounds exist but those are clunky at best.
Con Not a full DE
With a pure Xfce environment you cannot do as much as with Gnome or KDE.

Con Sessions cannot be disabled
There is a known bug where sessions keep getting saved involuntarily. So even when you try to clean your saved session it will be reproduced the next time you login.
Con Not good for different users' locales on one system
When you have users with different personal locales, XfcE has problems using the right locale for the right user.
Con Difficult to master
It does take time to learn the ins and outs of all that can be changed by editing the configs. It can also take some time to finally attain a configuration that is perfect.