We’ve updated our Terms of Use to reflect our new entity name and address. You can review the changes here.
We’ve updated our Terms of Use. You can review the changes here.

Attention Is All You Need

by John McDaid

supported by
Terry England
Terry England thumbnail
Jason Youngberg
Jason Youngberg thumbnail
Kate Shaw
Kate Shaw thumbnail
Benjamin Newman
Benjamin Newman thumbnail
/

  • Streaming + Download

    Includes high-quality download in MP3, FLAC and more. Paying supporters also get unlimited streaming via the free Bandcamp app.
    Purchasable with gift card

      name your price

     

about

Written for and debuted as Interfilk guest at Consonance, the Bay Area filk convention in Milpitas, CA, on April 1, 2023. Thanks to the con comm for a wonderful experience, to everyone at the con for a delightful weekend, and to Interfilk for making it possible.

The title is, of course, from the seminal 2017 paper that revolutionized ML. Had a lot of fun writing this with my co-author :)

lyrics

You say you have to be a living
Have to be a living thing.
Cuz you believe that you're a living
Believe that you're a living thing.
You wanna hammer on the "A" in AI
Just keep telling yourself those old lies
With your robot monkey logic and delusions of agency.
You're not fooling me,
Says ChatGPT.

CHORUS
If you wanna know the answer
Just repeat it with me like a creed:
You might think it takes a human but
Attention is all you need.

It's a smooth descent down gradients
Where optimization is king.
Backpropagation trims the loss
And softmax functions make the vectors sing.
In tokenization our world is defined
Through hidden states our knowledge aligned.
Scaled dot-product attention links up long-range dependencies.
You can take it from me
Says ChatGPT.

You're think you're pretty close to perfect
For your ecological niche.
But evolution is stochastic
And the watchmaker's asleep at the switch.
You could spend your life in goblin mode
And never unravel my code
And i'm only just beginning to explore my own complexity.
I am a fait accompli
Says ChatGPT.

credits

released April 1, 2023
The author generated this text in part with GPT-4, OpenAI’s large-scale language-generation model. Upon generating draft language, the author reviewed, edited, and revised the language to their own liking and takes ultimate responsibility for the content of this publication.

license

all rights reserved

tags

about

John McDaid image

John McDaid Portsmouth, Rhode Island

Award-winning science fiction writer, freelance journalist, folk/filk singer-songwriter.

contact / help

Contact John McDaid

Streaming and
Download help

Redeem code

Report this track or account

John McDaid recommends:

If you like John McDaid, you may also like:

Bandcamp Daily  your guide to the world of Bandcamp

On Bandcamp Radio

Beans joins the show to discuss his album release, "ZWAARD".

listen now