Revert the X11 window shaping

It did not build, xcb_shape_id was not resolving, and it was not worth fixing.
It bought a workaround for one configuration, cost two libraries and a window
grab fifteen times a second, and still gave harder edges than the thing it was
working around.

Qt's transparent window background is the right mechanism and it already works.
On X11 it needs a compositing manager, which Xfce ships with and which was
simply switched off. The warning that names that setting stays, so the cause
shows up in the log.

Kept from the reverted commit: opaque mode paints a neutral card rather than
the coat colour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

CMakeLists.txt

@@ -17,7 +17,6 @@ qt_add_executable(tncat
     src/catcontroller.h src/catcontroller.cpp
     src/appsettings.h src/appsettings.cpp
     src/platforminfo.h src/platforminfo.cpp
-    src/windowshape.h src/windowshape.cpp
 )
 
 target_include_directories(tncat PRIVATE src)
@@ -39,24 +38,6 @@ qt_add_qml_module(tncat
 target_link_libraries(tncat PRIVATE
     Qt6::Core Qt6::Gui Qt6::Quick Qt6::QuickShapes)
 
-# Without a compositor, X11 renders a transparent background as black. The
-# shape extension is the way out: it cuts the window down to the cat itself.
-# Both libraries ship with X11 and Qt's own xcb plugin already uses them.
-if(UNIX AND NOT APPLE)
-    find_package(PkgConfig QUIET)
-    if(PkgConfig_FOUND)
-        pkg_check_modules(TNCAT_XCB IMPORTED_TARGET xcb xcb-shape)
-    endif()
-    if(TARGET PkgConfig::TNCAT_XCB)
-        target_link_libraries(tncat PRIVATE PkgConfig::TNCAT_XCB)
-        target_compile_definitions(tncat PRIVATE TNCAT_HAVE_XCB_SHAPE)
-    else()
-        message(STATUS "TNCat: xcb-shape not found. Without a compositor the cat "
-                       "will sit in a black box; install libxcb-shape0-dev or "
-                       "xcb-util-dev and reconfigure.")
-    endif()
-endif()
-
 set_target_properties(tncat PROPERTIES
     MACOSX_BUNDLE TRUE
     MACOSX_BUNDLE_INFO_PLIST ${CMAKE_SOURCE_DIR}/Info.plist.in

README.md

@@ -99,24 +99,18 @@ input mask, which would make the overlay swallow every click on the desktop.
 
 This is the main target. Two things to know.
 
-**Transparency needs a compositor.** Without one, X11 has no alpha channel to
-composite against and the transparent window background renders as a black box
-around the cat. In Xfce it is one checkbox: Settings, Window Manager Tweaks,
-Compositor, Enable display compositing. That is the best answer, because alpha
-gives smooth edges.
+**Transparency needs a compositor.** Qt draws the cat on a transparent window
+background, and on X11 that only works while a compositing manager is running.
+Without one there is no alpha channel to blend against and the background comes
+out black, so the cat sits in a black box.
 
-When TNCat sees a window come up without an alpha channel it says so and falls
-back to shaping the window with the X11 shape extension: the window is cut down
-to the pixels the cat covers, so there is no box left to be black. The shape is
-one bit per pixel, so edges are harder than with a compositor, and it is
-refreshed 15 times a second to follow the animation. Measured on a sitting cat
-in a 120 x 106 window, the shape is 96 rectangles covering 29 percent of the
-window.
+Xfce has a compositor built in. Turn it on under Settings, Window Manager
+Tweaks, Compositor, Enable display compositing. That is the whole fix.
 
-Shaping needs `xcb` and `xcb-shape` at build time (`libxcb-shape0-dev` on
-Debian and Ubuntu, `xcb-util-devel` on SUSE). CMake says so if they are
-missing, and the build still works without them. `TNCAT_OPAQUE=1` skips the
-whole question and paints a solid background.
+TNCat prints a warning naming that setting when its window comes up without an
+alpha channel, so the cause is in the log rather than left to guesswork. If the
+compositor has to stay off, `TNCAT_OPAQUE=1` paints a solid background instead:
+the cat then sits on a plain card, which is honest but not pretty.
 
 **Keeping the cat on top.** The default is a frameless `Qt::Tool` window that
 stays on top. If the window manager puts it behind other windows or shows it in

qml/CatWindow.qml

@@ -50,17 +50,6 @@ Window {
     }
     Screen.onDesktopAvailableWidthChanged: cat.stageWidth = Screen.desktopAvailableWidth
 
-    // Without a compositor the window is cut to the cat's own outline. The
-    // drawing animates, so the shape has to be refreshed; 15 Hz is enough for
-    // a swinging leg and cheap enough for a window this small.
-    Timer {
-        interval: 66
-        repeat: true
-        running: Platform.shapeWindows && win.visible
-        triggeredOnStart: true
-        onTriggered: Platform.updateWindowShape(win)
-    }
-
     // The eyes need to know where the face sits on the desktop.
     Binding {
         target: win.cat

qml/ChipWindow.qml

@@ -27,14 +27,6 @@ Window {
         NumberAnimation { duration: 320; easing.type: Easing.OutQuad }
     }
 
-    Timer {
-        interval: 66
-        repeat: true
-        running: Platform.shapeWindows && win.visible
-        triggeredOnStart: true
-        onTriggered: Platform.updateWindowShape(win)
-    }
-
     Chip {
         id: curl
         anchors.fill: parent

qml/CupWindow.qml

@@ -22,14 +22,6 @@ Window {
            | (Platform.bypassWindowManager ? Qt.X11BypassWindowManagerHint : 0)
     visible: cat.cupVisible
 
-    Timer {
-        interval: 66
-        repeat: true
-        running: Platform.shapeWindows && win.visible
-        triggeredOnStart: true
-        onTriggered: Platform.updateWindowShape(win)
-    }
-
     Cup {
         id: mug
         anchors.fill: parent

src/platforminfo.cpp

@@ -1,7 +1,5 @@
 #include "platforminfo.h"
 
-#include "windowshape.h"
-
 #include <QGuiApplication>
 #include <QRegion>
 
@@ -20,34 +18,14 @@ void PlatformInfo::checkTranslucency(QQuickWindow *window)
     if (!window || m_opaque || m_warnedTranslucency)
         return;
     if (window->format().alphaBufferSize() > 0)
-        return; // a compositor is doing the blending, nothing to fix
+        return;
 
     m_warnedTranslucency = true;
-
-    if (WindowShape::supported()) {
-        m_shapeWindows = true;
-        emit shapeWindowsChanged();
-        qInfo("TNCat: no alpha channel on this window, which means no compositor "
-              "is running. Falling back to a shaped window. Edges will be a "
-              "little hard, because a shape is one bit per pixel. Turning the "
-              "compositor on gives smooth edges: in Xfce, Settings, Window "
-              "Manager Tweaks, Compositor, Enable display compositing.");
-        return;
-    }
-
     qWarning("TNCat: this window has no alpha channel, so the transparent "
-             "background renders as a black box, and window shaping is not "
-             "available either. On X11 that means no compositor is running. In "
-             "Xfce turn on Settings, Window Manager Tweaks, Compositor, Enable "
-             "display compositing. Or start TNCat with TNCAT_OPAQUE=1 to paint "
-             "a solid background instead.");
-}
-
-void PlatformInfo::updateWindowShape(QQuickWindow *window)
-{
-    if (!m_shapeWindows || !window || !window->isVisible())
-        return;
-    WindowShape::applyFromImage(window, window->grabWindow());
+             "background will render as a black box. On X11 that means no "
+             "compositor is running. In Xfce turn on Settings, Window Manager "
+             "Tweaks, Compositor, Enable display compositing. Or start TNCat "
+             "with TNCAT_OPAQUE=1 to paint a solid background instead.");
 }
 
 void PlatformInfo::setInputRegion(QQuickWindow *window, const QRectF &region)

src/platforminfo.h

@@ -21,7 +21,6 @@ class PlatformInfo : public QObject
     Q_PROPERTY(bool bypassWindowManager READ bypassWindowManager CONSTANT)
     Q_PROPERTY(bool opaqueBackground READ opaqueBackground CONSTANT)
     Q_PROPERTY(bool debug READ debug CONSTANT)
-    Q_PROPERTY(bool shapeWindows READ shapeWindows NOTIFY shapeWindowsChanged)
 
 public:
     explicit PlatformInfo(QObject *parent = nullptr);
@@ -47,23 +46,10 @@ public:
     // window that a window manager or a bad binding collapsed.
     bool debug() const { return m_debug; }
 
-    // True once a window has been seen without an alpha channel and the X11
-    // shape extension is there to make up for it. QML then keeps the window
-    // shaped to the cat instead of leaving a black box around it.
-    bool shapeWindows() const { return m_shapeWindows; }
-
-    // Looks at the window's format. Turns shaping on, or warns if there is no
-    // way to avoid the black box.
+    // Warns once if the window came up without an alpha channel, which is what
+    // a missing compositor looks like: transparency renders as black.
     Q_INVOKABLE void checkTranslucency(QQuickWindow *window);
 
-    // Grabs the window and cuts it down to the pixels the drawing covers.
-    Q_INVOKABLE void updateWindowShape(QQuickWindow *window);
-
-signals:
-    void shapeWindowsChanged();
-
-public:
-
     // Restricts the window's input region to one rectangle so clicks land on
     // whatever is behind the rest of the overlay. Implemented by the X11 and
     // Wayland plugins; a no-op on platforms that ignore the mask.
@@ -76,5 +62,4 @@ private:
     bool m_opaque = false;
     bool m_debug = false;
     bool m_warnedTranslucency = false;
-    bool m_shapeWindows = false;
 };

src/windowshape.cpp

@@ -1,143 +0,0 @@
-#include "windowshape.h"
-
-#include <QQuickWindow>
-#include <QtMath>
-
-#ifdef TNCAT_HAVE_XCB_SHAPE
-#include <QGuiApplication>
-#include <QtGui/qguiapplication_platform.h>
-#include <xcb/shape.h>
-#include <xcb/xcb.h>
-#endif
-
-namespace {
-
-// Anything this opaque counts as cat. Low on purpose: it keeps the soft edge
-// of an antialiased outline inside the shape instead of biting into it.
-constexpr int kAlphaThreshold = 24;
-
-} // namespace
-
-namespace WindowShape {
-
-// One rectangle per run of opaque pixels in a row. Neighbouring rows are not
-// merged: an X server copes fine with a few hundred rectangles at this size,
-// and merging them would cost more than it saves.
-QList<QRect> opaqueRuns(const QImage &image, qreal scale)
-{
-    QList<QRect> rects;
-    if (image.isNull())
-        return rects;
-    if (scale <= 0)
-        scale = 1.0;
-
-    const QImage rgba = image.format() == QImage::Format_ARGB32
-            ? image : image.convertToFormat(QImage::Format_ARGB32);
-
-    // The grab is in device pixels, the shape has to be in window coordinates.
-    // Every device row that falls into a window row is folded into it, so a
-    // pixel that is opaque in either of them keeps its place in the shape.
-    const int cols = qMax(1, qCeil(rgba.width() / scale));
-    const int rows = qMax(1, qCeil(rgba.height() / scale));
-    QList<bool> row(cols);
-
-    for (int ly = 0; ly < rows; ++ly) {
-        row.fill(false);
-        const int first = qFloor(ly * scale);
-        const int last = qMin(rgba.height(), qCeil((ly + 1) * scale));
-        for (int y = first; y < last; ++y) {
-            const QRgb *line = reinterpret_cast<const QRgb *>(rgba.constScanLine(y));
-            for (int x = 0; x < rgba.width(); ++x) {
-                if (qAlpha(line[x]) >= kAlphaThreshold)
-                    row[qMin(cols - 1, int(x / scale))] = true;
-            }
-        }
-
-        int runStart = -1;
-        for (int lx = 0; lx <= cols; ++lx) {
-            const bool opaque = lx < cols && row[lx];
-            if (opaque && runStart < 0)
-                runStart = lx;
-            else if (!opaque && runStart >= 0) {
-                rects.append(QRect(runStart, ly, lx - runStart, 1));
-                runStart = -1;
-            }
-        }
-    }
-    return rects;
-}
-
-#ifdef TNCAT_HAVE_XCB_SHAPE
-
-namespace {
-
-xcb_connection_t *connection()
-{
-    const auto *x11 = qGuiApp->nativeInterface<QNativeInterface::QX11Application>();
-    return x11 ? x11->connection() : nullptr;
-}
-
-bool haveShapeExtension(xcb_connection_t *conn)
-{
-    static int cached = -1;
-    if (cached >= 0)
-        return cached == 1;
-
-    const xcb_query_extension_reply_t *ext = xcb_get_extension_data(conn, &xcb_shape_id);
-    cached = (ext && ext->present) ? 1 : 0;
-    return cached == 1;
-}
-
-} // namespace
-
-bool supported()
-{
-    xcb_connection_t *conn = connection();
-    return conn && haveShapeExtension(conn);
-}
-
-bool applyFromImage(QQuickWindow *window, const QImage &image)
-{
-    xcb_connection_t *conn = connection();
-    if (!window || !conn || !haveShapeExtension(conn))
-        return false;
-
-    const QList<QRect> runs = opaqueRuns(image, image.devicePixelRatio());
-    if (runs.isEmpty())
-        return false;
-
-    QList<xcb_rectangle_t> rects;
-    rects.reserve(runs.size());
-    for (const QRect &r : runs) {
-        rects.append({ static_cast<int16_t>(r.x()), static_cast<int16_t>(r.y()),
-                       static_cast<uint16_t>(r.width()), static_cast<uint16_t>(r.height()) });
-    }
-
-    xcb_shape_rectangles(conn, XCB_SHAPE_SO_SET, XCB_SHAPE_SK_BOUNDING,
-                         XCB_CLIP_ORDERING_UNSORTED,
-                         static_cast<xcb_window_t>(window->winId()), 0, 0,
-                         rects.size(), rects.constData());
-    xcb_flush(conn);
-    return true;
-}
-
-void clear(QQuickWindow *window)
-{
-    xcb_connection_t *conn = connection();
-    if (!window || !conn || !haveShapeExtension(conn))
-        return;
-
-    xcb_shape_mask(conn, XCB_SHAPE_SO_SET, XCB_SHAPE_SK_BOUNDING,
-                   static_cast<xcb_window_t>(window->winId()), 0, 0, XCB_PIXMAP_NONE);
-    xcb_flush(conn);
-}
-
-#else // no xcb-shape at build time
-
-bool supported() { return false; }
-bool applyFromImage(QQuickWindow *, const QImage &) { return false; }
-void clear(QQuickWindow *) {}
-
-#endif
-
-} // namespace WindowShape

src/windowshape.h

@@ -1,33 +0,0 @@
-#pragma once
-
-#include <QImage>
-#include <QList>
-#include <QRect>
-
-class QQuickWindow;
-
-// Cuts the window down to the pixels the cat actually covers, using the X11
-// shape extension. Without a compositor X11 has no alpha to blend against and
-// a transparent background comes out black, so the only way to lose the box
-// around the cat is to make the window itself that shape.
-//
-// Everywhere else this is a no-op: with a compositor, plain alpha is both
-// cheaper and smoother, since a shape is one bit per pixel and cannot
-// antialias an edge.
-namespace WindowShape {
-
-// Runs of opaque pixels, one rectangle per run, in window coordinates.
-// Platform independent so it can be tested without an X server.
-QList<QRect> opaqueRuns(const QImage &image, qreal scale);
-
-// True when the platform can shape windows and the build has the pieces.
-bool supported();
-
-// Shapes the window to the opaque pixels of the image, which is expected to be
-// a grab of that same window. Returns false if nothing was applied.
-bool applyFromImage(QQuickWindow *window, const QImage &image);
-
-// Drops the shape again, leaving a plain rectangular window.
-void clear(QQuickWindow *window);
-
-} // namespace WindowShape