Clojure and Firestore and Cloud Functions, oh my

Findka

The first batch of recommendations was emailed out last Friday. Exciting times. I enjoyed going through the content items that users submitted. About 20 users signed up (and you had to pick at least three seed content items to sign up). I’ve even re-done the landing page so it doesn’t look quite so crappy.

Since there aren’t many users yet, a decent chunk of the recommendations were things that I put in the system myself. After perusing the thumbs up/thumbs down events that users generated, I was sad to see that no one liked my Nate the Snake recommendation. Same for the Shia LaBeouf - Just Do it remix. I guess they’re an acquired taste.

Most of my work now will be going into marketing and growth. I’d like to find ways to encourage users to refer their friends, and I need to figure out what the best way is to manually recruit new users. I submitted my Y Combinator application last week as well. Hopefully third time’s the charm. Regardless, I’ve been feeling very confident about Findka since the most recent directional change a couple weeks ago, so I’m happy whether I get in or not.

During all this, I got to use Firebase’s Firestore and Cloud Functions for the first time. I was pleased with both of them:

Firestore

I originally dismissed using Firebase for my apps’ primary data store because “NoSQL, gross”—I like having normalization and a query language. However, you can still store normalized data within a hierarchical store, similar to how Fulcro organizes its client-side data. So I thought I’d try out Firestore and see how far it can go before needing to switch to Datomic or something similar.

It turns out that Firestore actually does encourage you to store your data somewhat normalized—it’s not just a plain JSON document like I believe the old Realtime database was.

Firestore’s data model has basically three parts:

• Collections

• Subcollections

• Nested data

Here’s an example of what a bunch of Firestore data might look like if you mapped it on to Clojure data structures:

{:users
 {"dGhlI" {:email "john@example.com"
           :likes-tennis true
           :address {:line1 "123 Oak blvd."
                     :country "Elbonia"}
           :events {"HF1aW" {:type "like"
                             :item [:items "NrIGJ"]
                             :timestamp #inst "2020-03-02"}
                    "yb3du" {:type "dislike"
                             :item [:items "IGZve"]
                             :timestamp #inst "2020-03-03"}}}
  "CBqdW" {:email "suzy@example.com"
           :events {"yb3du" {:type "like"
                             :item [:items "IGZve"]
                             :timestamp #inst "2020-03-01"}}}}
 :items
 {"NrIGJ" {:name "Cheese"
           :url "https://cheese.com"}
  "IGZve" {:name "Lotion"
           :url "https://lotion.com"}}}

Notes:

• The are two collections: users and items. Each key (e.g. "dGhlI") is a string that Firebase randomly generates. Real keys are much longer than five characters. You can set keys explicitly if you want, which is useful for some kinds of data.

• :address is an example of nested data. You can nest maps and vectors arbitrarily, though you should of course break nested data off into separate collections instead when it makes sense.

• :events is an example of a subcollection. Subcollections are basically just regular collections that give you a convenient way to include some other entity in the key. Comparing it to Datomic, subcollections are like including a cardinality-many component attribute. You can nest subcollections.

• The :item key is a reference to another entity. These can dangle; deleting the referenced item entity won’t remove the :item key from the event entity. You could also have a vector of references (i.e. to-many relationships).

So modeling data isn’t bad actually. Alas, the queries are pretty weak: they give you filtering, sorting and pagination, but that’s pretty much it. No joins. But in return, the queries are efficient and subscribable. Firestore maintains indexes for each attribute on each entity, and queries are limited to whatever can be looked up efficiently with the indexes.

I’ve written some code that, when the app loads, takes a vector of queries and loads the results into an atom which I then use to drive the UI. To handle complex queries, I use Firebase queries to load all relevant documents, and then map/filter/join/etc over the data after it’s in the atom. For the simple things I’ve needed to do so far, this has worked out alright.

I haven’t yet done anything with subscribing to queries. I’m planning this week to write a Clojure wrapper over Firestore, and as part of that I’d like to make it so you can associate queries with Rum components. When the components mount and unmount, they’ll subscribe and unsubscribe to their queries. Or maybe you just specify subscriptions at the app level and don’t worry about unsubscribing.

There’s a benefit to query subscriptions even if your app isn’t multi-user: you can let Firebase handle pending changes. I haven’t tested this yet, but I think I remember reading that as soon as you write to Firestore, the client SDK will commit the change locally (optimistically, with a pending flag) and notify any subscriptions that would be affected. Presumably Firestore will roll it back if the write fails.

So there’s a lot of fun stuff that can be done with Firestore—I’m excited to see how this wrapper library turns out.

Cloud Functions

I don’t have as much to say about these except that it was surprisingly easy to write them with ClojureScript. I used these two examples. ShadowCLJS sure is nice; just throw in another build config section.

Unfortunately, I haven’t figured out how to access the right execution context while debugging. To illustrate, consider a function like the following:

(defn handle-request [req]
  (def req req)
  ...)

Normally I’d set something like that up (typically with scope-capture) and then execute e.g. (pprint req) from Vim (using Fireplace). This works fine in normal, web CLJS apps. But when I do it in a Functions context, it never works. Whatever execution context my editor is connected to, it isn’t the one that the function used. My guess is that each function execution gets a new context. I don’t really know how it works under the hood.

So far I’ve made due, but it’d be really nice to know if there’s a way to fix that.

Clojure guide

Again, nothing new this week. But all the stuff I’ve been doing with Firestore and Functions has given me lots of material to work with for upcoming posts. Originally I was planning to include only a cursory introduction to Firestore and then quickly switch over to Datomic. However, I now think that a lot of apps would do just fine with Firestore—and that certainly involves less overhead than Datomic. So I’ll probably see how far we can get in the guide with just Firestore. Maybe I’ll write the whole application with Firestore and then re-write it using Datomic (or Crux, or something).

Miscellaneous

I’m helping to organize a new Clojure meetup here in Provo—so for anyone in the area, be sure to join the Meetup group. There’s also #clojure-utah on Clojurians Slack.