Copy & Paste, Drag & Drop
Interaction Techniques and Technologies (ITT), SS 2025
Raphael Wimmer
Overview
These are slides/notes for the lecture, automatically generated from the slide set. Please extend this outline with your own notes.
Common Interaction Techniques and Implementations
Shneiderman's Eight Golden Rules of Interface Design
- Strive for consistency.
- Enable frequent users to use shortcuts.
- Offer informative feedback.
- Design dialog to yield closure.
- Offer simple error handling.
- Permit easy reversal of actions.
- Support internal locus of control.
- Reduce short-term memory load.
<small>Shneiderman, B. and Plaisant, C., Designing the User Interface: Strategies for Effective Human-Computer Interaction: Fifth Edition, Addison-Wesley Publ. Co., Reading, MA (2010), 606 pages. online </small>
Overview
- Clipboard, Drag and Drop
- User Interfaces
- Interactive exploration
- Implementations in different operating systems
- Clipboard managers
Clipboard
General Concept
Questions:
- What is a clipboard?
- How does the user interface for clipboards look like?
- How can clipboard functionality be implemented?
Buzz Group
How does a clipboard behave? What do you expect from a typical clipboard?
Clipboard Usage
- 3-4 interactions with the clipboard per hour (maximum: 20)[^stolee]
- programmers: 16 interactions / hour[^kim]
- copy: 45%, paste: 52%, cut: 3%[^stolee]
- 80% of copied data is pasted later[^stolee]
- majority of copy/paste operations restricted to ca. 5 applications per user[^stolee]
- most copied from: browser[^stolee]
- most pasted to: text editor[^stolee]
- clipboard mostly accessed via shortcuts[^retzer]
- context menu rarely used (exception: copying images)[^retzer]
[^stolee]: Stolee, Elbaum, & Rothermel (2009) Revealing the Copy and Paste Habits of End Users. [^retzer]: David Retzer (2017) “Implementierung eines Copy&Paste-Mechanismus mit Quellenattribuierung” (Master's Thesis). [^kim]: Kim, Bergman, Lau, & Notkin (2004) An Ethnographic Study of Copy and Paste Programming Practices in OOPL.
Clipboard Usage Patterns
[^stolee]: Stolee, Elbaum, & Rothermel (2009) Revealing the Copy and Paste Habits of End Users
Buzz Group
How does a clipboard work?
Qt Clipboard inspector
~~~~
!/usr/bin/python3
from PyQt6.QtWidgets import QApplication
app = QApplication([“”,“”]) clipboard = app.clipboard() mimeData = clipboard.mimeData() print(“Current clipboard offers formats: ” + str(mimeData.formats())) for f in mimeData.formats():
print("---- %s ----" % f) data = str(mimeData.data(f)) if len(data) > 100: print(data[:100] + " [...]") else: print(data) print("")
~~~~
Microsoft Windows
- Clipboard formats.aspx) describe the type of data in the clipboard
- Clipboard may contain the same data in different formats (e.g., as plain text, HTML, RTF)
- Standard formats.aspx) (CFBITMAP, CFUNICODETEXT, …)
- Registered formats (custom, for data exchange between applications, e.g., CF_HTML.aspx)
- Private formats (for internal use)
- Access via SetClipBoardData(format, data), IsClipBoardFormatAvailable(format).aspx), EnumClipboardFormats.aspx), GetClipBoardData().aspx)
- Clipboard needs to be explicitly cleared before copying data to it.
Mac OS X / macos
- “Pasteboard” for historical reasons
- manager process
pbs
accessed via APIs - Standard classes for clipboard content (
NSString, NSAttributedString, NSImage, NSURL, NSColor, NSSound
), general classNSPasteboardItem
or custom classes adoptingNSPasteBoardReading
protocol - additional find buffer for text searches
- see also: pasteboard programming guide
iOS
- General Pasteboard and Find Pasteboard similar to macos
- apps can create additional pasteboards
Uniform Type Identifier
(UTI) describes type of data (e.g.,public.jpeg
orcom.myCompany.myApp.myType
)Pasteboard
class provides convenience methods for copying/pasting images, URLs, …
X11 (Linux/BSD)
- Main difference to OS X and Windows: no data is actually stored in the clipboard
- Instead:
- On “copying” data to the clipboard, the application tells the X Server that it now owns the
CLIPBOARD
selection. - The previous owner of the
CLIPBOARD
selection gets notified. - When pasting data from the clipboard, the application directly requests the data from the application that owns the
CLIPBOARD
selection. - Content negotiation: pasting applications asks for data types the application can provide and can also try to request data in arbitrary format.
- furthermore:
PRIMARY
buffer (holds selected text, pasted via middle click) andSECONDARY
buffer (mostly unused), similar mode of operation as clipboard. - data types identified via MIME types
Further reading: explanation by Jamie Zawinski, Freedesktop.org clipboard "specification", ICCCM: Peer-to-Peer Communication by Means of Selections, X11: How does “the” clipboard work?
Wayland (Linux)
- Wayland: successor to X11 protocol
- Similar to X11: source application creates a
wl_data_source
, receiving application sees it aswl_data_offer
object. - Source application creates
offer
for each MIME type that is offered. - Receiving application sends selected MIME type and file descriptor to source application.
- Source application writes data into file descriptor.
Further reading: Wayland protocol: data sharing, Wayland clipboard and drag & drop
Android
- Clipboard holds one
ClipData
object at a time, consisting of:ClipDescription
object containing list of MIME types- one or more
ClipData.Item
s, all having the same type: text, URI, or Intent
- Content providers allow for retrieving data with a specific MIME type via a URI
- Usage:
[ClipboardManager](http://developer.android.com/reference/android/content/ClipboardManager.html).setPrimaryClip(ClipData)
Qt Implementation
- data type indicated via MIME
Clipboard Managers
- listen to changes in clipboard state (X11: needs to grab ownership of
CLIPBOARD
selection) - add additional features
- persist history
- select format to paste
- modify data
- add formats (e.g., OCR)
- shortcuts for managing clipboard contents
- …
- See also: CopyQ, Wikipedia: Clipboard Manager
Further Reading
- "Copy-paste conclusions: put the metadata on the clipboard"
- “Step by step we’ve been teasing apart the clipboard to see how metadata could survive a copy-paste between applications. If the metadata survives, then the destination can be used to automatically credit the original source and the creator, without the user having to do it manually.” "On Clipboard Formats – 2006-09-15"
- “The Carbon version of Gecko doesn’t interoperate with anything but other Carbon Gecko processes. I figured I should try to do better with the Cocoa nsClipboard.
This stuff is so underdocumented that it isn’t even funny. This document is written so that others might find something when they search the Web.”*
Drag and Drop
- X11: DND extension
- Windows: Object Linking and Embedding (OLE)
- OS X: Cocoa Drag Manager
- generally similar to clipboard operation
- source application may offer multiple formats
- on drag start: change cursor to indicate dragged object
- when dragging into another window: receiving window can accept or reject drag
- data is only copied on release of mouse button
DnD Qt Implementation
- again: wrappers with OS-specific code
- MIME-encoded content (like X11), gets converted for Windows and OS X.
- Code example: tutorial at zetcode.com and
dragdrop_example.py
in GRIPS.
Your Tasks
- Try out the clipboard and drag-and-drop examples
- Find out what kind of data different applications offer on different operating systems
- Write a small tool that intelligently modifies the clipboard contents (e.g., removing all passwords from plain text)
Recap
- Different clipboard implementations across operating systems
- Data often available in different formats
- X11, Wayland have an asynchronous clipboard
- format negotiation in X11 is quite a mess