16
64 Comments

🚀 Just launched a new scalable cloud graph database! Any feedback/users/testers appreciated!

My brother and I have just launched our scalable cloud graph database SurrealDB in public open beta (completely free to use)! We've been working on this for a number of years, and are completely bootstrapped.

The first beta version supports a large amount of functionality, with the ability to run in a highly-available, highly-scalable distributed environment. A highly-performant query language enables use of many different data types, and embedded JavaScript functions can be used for more complex functions and triggers.

We wanted to create a database that people didn't have to manage, so they could focus on building applications, not the infrastructure. We wanted users to be able to use schema-less and schema-full data patterns effortlessly, a database to operate like a relational database (without the JOINs), but with the same functionality as the best document and graph databases. And with security and access permissions to be handled right within the database itself. We wanted users to be able to build modern real-time applications effortlessly - right from Chrome, Edge, or Safari. No more complicated backends.

I welcome any feedback! Thank you very much in advance.

  1. 3

    The database looks really cool and I'd love to try it out in a weekend project! You're saying that the database is currently in public beta. Do you have any pointers to how production ready the database is at the moment? I'm thinking solid architecture combined with real world usage are necessary to achieve maturity.

    May I ask how you are doing the illustrations on your website? They look really awesome.

    1. 2

      Thank you very much for the kind words @thmsobrmlr! I have created all the illustrations on the website using Photoshop.

    2. 2

      Hey @thmsobrmlr thanks for your comment! Yeah it's currently in public beta. So we've been running SurrealDB (the pre-beta Golang version) in production since 2019. We learnt a lot from building a database in Golang, and so the Rust version has many more improvements (memory improvements, memory safety, performance improvements, query language improvements, data serialisation improvements, and so many more things...).

      Some of the SaaS platforms that we are running on SurrealDB can be seen here: https://github.com/surrealdb/showcase . There are definitely performance improvements we want to make SurrealDB, and we'll be making these as soon as possible, before our 1.0 launch (https://surrealdb.com/roadmap). We will also be launching some demo/example apps which will help with getting started on SurrealDB.

      We'd love for you to try SurrealDB out on a weekend project! If you have any questions then you can reach us on Github / Discord (https://surrealdb.com/community)!

      The website/branding/design/illustration wizardry is all @jscmh so I'll let him answer that 😀!

  2. 3

    love this idea. I wonder how to apply it to my next ETL project

    1. 2

      Thanks @makeshiftIndie! Would be great to see how SurrealDB could be used in a full ETL project. As @jscmh said - feel free to ask any questions, or if you have any suggestions we'd love to hear them!

    2. 2

      Thank you very much @makeshiftIndie! 😊 Jump on our Discord (https://surrealdb.com/discord) so we can answer any questions you might have.

  3. 3

    Congrats, maybe also consider posting on BetaList (to nab a few more testers)

    1. 2

      Thank you vey much @MrJustinF! Submitted to BetaList!

    1. 2

      Thank you very much @erengunduz! 😊

  4. 3

    Wow that's hugely ambitious!! Your landing page looks amazing.
    Best of luck, look forward to following your progress.

    1. 2

      Thank you very much indeed @mr_olafsson!

    2. 2

      Thanks for your kind words @mr_olafsson! We'll be keeping our Indie Hackers profile up-to-date, but you can also follow the community in other ways if you want (https://surrealdb.com/community)!

  5. 3

    The site looks really nice, good luck with the launch!

    Do you have any example apps that use the db? I saw the showcase on GitHub but didn't see any examples in the repos.

    1. 2

      Thanks @leesus! As @jscmh said we're looking to get some example apps as soon as possible! We'll let you know when we've got something ready!

    2. 2

      Hi @leesus , thanks for your comment! We know we are lacking in this area at the moment, and are looking to add some example / demo applications soon (React and Ember.js to begin with). We also plan to release a few videos to show just how quick and easy it can be to build on top of SurrealDB.

  6. 3

    The page looks 🤯
    Very nice approach. Congrats

    1. 2

      Thank you very much @fabcgn! Glad you like it!

  7. 3

    Looks real polished! best of luck

  8. 3

    The site looks great and the docs are sooo clean... well done! I will definitely look into using this when I start on the dev work for my project!

    1. 3

      @jwalker thanks so much! When you're ready, you can join our Discord (https://surrealdb.com/discord) for questions, help, and for any ideas you might have!

    2. 3

      Thank you very much indeed @jwalker!

  9. 2

    What was the motivation to creare a DB??there are plethora of DB’s available .
    What problem are you tragetting to solve?!
    Kudos on the DB though.
    Would love to see tutorials or blogs on the internal workings of the DB.
    May be I can create one tutorial or blog for this.

    1. 1

      Thank you very much for the kudos @gauravnlotekar 👍

    2. 1

      Hi @gauravnlotekar! Thanks for your comment! There are definitely loads of databases out there and we love so many things about pretty much all of them! There wasn't one we were truly happy with though!

      I think the main thing that was super important to us was the ability to build our frontend applications super easily and quickly. We used to build our applications on top of PostgreSQL + OrientDB + InfluxDB + DynamoDB (and we also have had MongoDB / RethinkDB / Couchbase in our stacks too), but we were always building complicated APIs in front of these databases in order to get the functionality that we wanted in our apps. SurrealDB enables us to cut out all of this development and infrastructure management, so that we could focus on the features in the app and not the infrastructure. it also means we are able to build better apps, and build them quicker.

      We also didn't want to rely on an API in the cloud, or a data-layer (not quite a database but something that enabled storing and querying data aka Firebase). We wanted a full-blown database so we could query our data in many different flexible ways (computed views for analytics dashboards, graph connections for relationship analysis, nested fields for complex document data) - directly from the client.

      In addition there were so many nice features from different separate databases that we wanted in one platform. Computed analytics views in CouchDB and Couchbase, nested fields and documents in MongoDB, remote http requests in RethinkDB, sql graph querying in OrientDB, permissions and security in Firebase, distributed in CockroachDB, schemafull in relational databases, a full-on type system in relational databases, graph relations in Neo4j, and so on... With all of these features, one can build some pretty powerful frontend apps!

      What we've seen so far is that the developers who are getting started on SurrealDB are all using it in different ways - which is great - so it's going to be interesting working out what people love about it most!

      We're definitely going to be releasing tutorials and blogs on the internal workings of the DB, and we have some blog posts planned for a closer look at the Rust ❤️ code, and some of the libraries that we have used.

      It would be absolutely brilliant for any contributions with regards to tutorials or blogs! That would be amazing! Out of interest, what languages/frameworks/infrastructure/tools is your expertise in?

  10. 2

    This is neat. Curious- how does it handle scaled data? From billions to trillions, daily.

    A lot of these newer computer/serverless projects are wicked fast until they hit some local maximum.

    1. 1

      Hi @productpirate, thanks for your comment! As this is our initial beta release at the moment, there are a number of code performance improvements that we will be getting to as soon as possible. As soon as those improvements are done, we will get on to some proper benchmarking of massive datasets, and will get these on to the website!

      In terms of scalability of requests, it is possible to horizontally scale SurrealDB so that more, and more data compute nodes are able to handle more data. Database transactions are concurrent and will only conflict if data to the same record is made at the same time. In this respect it is able to scale to really large data sets.

      We are also working on a new key-value store implementation for single-node use-case. We've chosen not to go with RocksDB/LevelDB or other libraries, and are building our own key-value store (which will be open source). The technical design of this kv store will be really performant, and it will will operate in a hybrid of memory and on-disk (all datastore keys will be in memory). I actually wrote a thesis / white paper on this (https://surrealdb.com/static/whitepaper.pdf).

      We've got some exciting things to come in the pipeline!

  11. 2

    Sound like OrientDB. How will it be different?

    1. 2

      Hi @Tzahi, we were actually big users of OrientDB way back in 2014-2015, and it was one of the reasons we decided to create SurrealDB. We took a lot of inspiration from OrientDB (you'll notice the similarities, and differences, with the SQL syntax and RIDs). We've also taken inspiration (albeit somewhat different) from CouchDB, MongoDB, InfluxDB, RethinkDB, Firebase, CockroachDB... (we love databases)!

      There are a number of differences between SurrealDB and OrientDB:

      1. Main difference is that you can connect directly to SurrealDB from a client device (SurrealDB has permissions, and authentication built right into it so you can securely run queries directly from Chrome/Safari or an Electron app). This is similar to Firebase.

      2. Differences in the query language. We have implemented record ids differently, so that they can be used easily directly from clients, and play nicely with URLs. There are also differences with the query language (we've spent a lot of time trying to make the SurrealQL query language as nice to use as possible). We have in-built JavaScript functions, computed variables (which only compute when data is selected), decimal numbers (so you can easily perform mathematical calculations right in the browser without experiencing 64 point floating number rounding issues), geospatial/geojson types, complex expressions....

      SELECT ->knows->(? AS f1)->knows->(? AS f2)->(knows, likes AS e3 WHERE influencer = true)->(? AS f3) FROM person:tobie;

      1. There is a difference in how the distributed mode works in SurrealDB. SurrealDB is built on top of ACID key-value stores.

      2. We are close to releasing our WebAssembly release, which will enable SurrealDB to run in the browser, using IndexedDB as a persistent backing store. Eventually, this in-browser version will sync to the cloud allowing for offline and online data syncing for applications.

      If you haven't already seen it, our features page (https://surrealdb.com/features) is a great place to see an overview of a lot of the functionality.

      Hope that answered your question. Feel free to ask anything else!

      1. 2

        This is indeed different. Sounds like an adaptor for data stores.
        The main issue with orient which i use for the last few years for a customer is the corruptions it generates and inconsistencies in transaction handling in the embedded version to the remote version.

        1. 3

          @Tzahi I wouldn't want to criticise any other database as we have learnt a lot from using other databases over the years, and love so many features from so many of the popular databases - but yes this was a key frustration for us.

          SurrealDB is built in Rust, and the SurrealDB server itself (web server and API server) actually uses the embedded version for all of its functionality. So our embedded version (we'll be launching a WebAssembly embedded version, Python native embedded version, NodeJS native version, and C native version) operates in exactly the same way as the distributed cloud version.

          We've also spent a great deal of time making sure SurrealDB gives consistent responses so that you don't get surprised by certain things not working as you expect them to do so!

          Let me know if you have any other questions!

  12. 2

    How does this compare to Neo4j or their Aura solution?
    My company is mainly building big data solutions on Neo4j for enterprise clients. And their biggest concern is that they have a "big" provider in the back which will support that graph database in the future.
    Checked your QL it does look much simpler compared to Neo4j at least for inserting data.
    And all clients really love certifications for graph databases :)

    1. 3

      Hey @ivanderbu2. We'll be launching our fully managed, cloud-hosted platform later this year - which will be comparable to the Aura hosted cloud version. In the meantime you can download and run SurrealDB locally, or in your own cloud. - we'll have some documentation for deploying SurrealDB in the production soon.

      We've spent a lot of time trying to make SurrealQL simple and beautiful to use. You can do some really clever stuff with the language, and extend it with JavaScript if you want to go even further!

      I think a big difference between SurrealDB and Neo4j is the table concept with the graph edges between. It makes it very simple to model your data (thinking in terms of tables and types), but with the addition of simple record links, and full graph edges.

      We have a very big vision for SurrealDB, and have a lot of stuff planned for the future. Being able to support enterprise clients is obviously of big importance to us, and especially as we grow, that's going to be a key focus for us. We aren't going anywhere! In addition SurrealDB is effectively open source. There are no limits to using the database in an enterprise setting either in single-mode or distributed mode. The license allows everything except for setting up a paid cloud-hosted platform. So from an enterprise use-case point-of-view, it is effectively open source, and has no limits.

      Can I clarify what you mean by certifications? Certifications for developers, or the platform type (ISO 27001, SOC 2, HIPAA)?

      1. 1

        Hey Tobie thank you for such a detailed response.
        I'll for sure give it a try and test it.

        For certification I meant for developers. That's giving some assurance to enterprise clients that they can easily find new verified talent if needed. If talent pool is small they tend to skip those projects.

        I know it's super complex to have all the pieces when you just launched it. I've no doubt that there is a huge demand for solution like SurrealDB.

  13. 2

    congratulations! granularity of permissions is table right? and not fields within the table? also are the hooks available before crud operations?

    1. 2

      Thanks @pdyc! Granularity of permissions is on TABLEs and FIELDs.

      DEFINE TABLE person SCHEMAFULL PERMISSIONS FOR select WHERE $auth.admin = true OR author = $auth.id;
      and
      DEFINE FIELD metadata.hidden ON person TYPE object PERMISSIONS WHERE $auth.admin = true;

      1. 2

        wow that's great thanks for the example.

        1. 2

          @pdyc just noticed that I didn't reply about hook on CRUD operations...

          So in SurrealDB when a record is created, updated, or deleted then an event can be run. Unlike in MySQL/PostgreSQL these events only happen ONCE the data has been edited. However, they are pretty advanced, so here are a couple of examples of what you can do...

          Add a record to another table pointing back to the record which was updated...
          DEFINE EVENT changed ON person WHEN $event = 'UPDATE' THEN ( CREATE activity SET action = 'updated', person = $this, time = time::now() );

          Call a remote webhook if an email address field is changed on a record...
          DEFINE EVENT changed ON person WHEN $before.email != $after.email THEN http::post('https://my-webhook.com', $after);

          Hope that answers your question fully!

          1. 2

            yes however my weekend is ruined since i will be playing with surrealdb this weekend :-)

            1. 2

              Sorry! 😂😂😂

            2. 2

              🤣 We apologise @pdyc! 🤣

  14. 2

    How can I trust your database is reliable and more or less as bullet proof as PostGRE, MariaDB/MyQL? Do you or your team have background in developing database solutions? Or worked for one on the major vendors and decided to branch off?

    No matter how cool the database may be - no way can I really use it any real sense until I have faith in the technology. Data migrations take a lot of work.

    1. 3

      Hi @Ghidra, that's a really good point / question. So obviously it's early days (this is only our initial 1.0 beta release), while MySQL was released in 1995, PostgreSQL in 1996, MongoDB in 2009, and so on, as you know! However we have been building SurrealDB over the last 7 years, and have been using it in production (at scale) in a number of SaaS and ecommerce platforms for the last 4 years, while we continued to build it out and develop the features we needed. We both have vast experience building distributed and highly-available software platforms, with a large variety of backends / setups / databases / technologies - and that was the very reason why we decided to build SurrealDB - so that we could simplify that very process.

      We know there are performance improvements needed in certain parts of the database code (especially with zero-copy datastore key serialization, and also with certain reference sharing within the code), but we have these changes planned!

      In addition, I've completed a MSc in Computer Science / Software Engineering from University of Oxford where I focused on distributed systems and databases. My distinction thesis (https://surrealdb.com/static/whitepaper.pdf) focused on a very core part of the underlying technology (key-value stores) for use in distributed databases.

      Finally, we initially built our database (for ourselves) in Golang, however after extensive experience pushing the language to its limits, and overcoming some of the difficulties with the language, when open sourcing it we decided to re-build it in Rust. This obviously has a number of benefits including memory safety, performance, and a number of improvements to the SQL query parser and serialisation technology, which translate to a much more stable product!

      1. 2

        Thanks for your reply! Very cool stuff for sure; it looks like a very interesting + useful product.

        I don't know how prevalent my concern is with other early adopters, but it may help to to get the reply you wrote above across on the landing page - especially the part about it being 7+ years old and used in production at scale at a number of SaaS / ecommerce companies. I did not get the message from the landing page which is what led me to raise it here. (I thought it is BRAND BRAND new which made me concerned about issues)

        Best of luck!

        1. 2

          Thank you very much @Ghidra! VERY good point. We will add this to the site ASAP.

  15. 2

    After a few years work with almost every db out there, like dbase, paradox .... Yes before 2000 in my first adventures, mysql, postgres, mongodb, neo4j, sqlserver (my old boss force me, I hate it), scylladb, you get the point, and many others like csv ahahahah

    I must say one only word this db is surreal, or better this is surrealdb

    I started work with it, dirt my hands, and this is something from other galaxy, is the ddd (dev dream database), is so versatile that I dont have words, we can use schema, noschema docs, graphs, code, functions, previleges, auth, events, javascript, variables etc etc its to powerful (and i'm only start with it), is better to check it out asap, will not regret

    I have only one problem with it,
    Its fuc.....my vacancies (ok sun is hidden here in portugal this week)
    ....because I cant stop play with it, read the docs, create some models and scheme, using it, and this is good stuff, and i just touch the tip of iceberg yet

    Even a mature developer with years of database, have a hard learning curve with it with the more advanced stuff because of the power of it, we can do almost anything, and is 100% restful, you can even simply connect your frontend or simply curl and thats all

    Im enjoy it, try for yourself and say that im wrong

    When we boot the server in 1ms and check the size of the binnary 20mb (no runtime, rust powered) and remember sqlserver+sql managment studio install file more that 1.5gb we think about, whats is all about, WTF! its real?

    Thanks Tobie and Jamie and whole team for this Master Piece

    All of above is my humble opinion after only a few days with it, get your own opinion for yourself

    Sorry my english......its not my native language

    1. 2

      @koakh thanks for your awesome comment 😀! Great to have you on-board the SurrealDB journey!

    2. 2

      Thank you very much for the kind words @koakh! Your English is great! @tobiemh and I have many exciting features coming to SurrealDB. Thank you very much for all your feedback and contribution so far!

  16. 1

    Thank you very much for all your great feedback!

  17. 3

    This comment was deleted 10 months ago.

    1. 2

      Hi @ShaneBellone - thanks for your comment! In addition to the query language being very different, it also has permissions and realtime functionality built right into every table/field, along with authentication, so you can connect to it and query directly from a browser, client library, or end user device.

      Furthermore, you don't have to choose upfront how you store your data (i.e you don't have to choose between tables, documents, links, or graph, and you don't have to choose a specific API - but can use them all at the same time), opening up some interesting possibilities when it comes to modelling and analysing your data.

      In addition to the distributed cloud version, we plan to release an embedded version soon too, which will enable embedding SurrealDB in the browser (in WASM), in Python, Node.js, and C to begin with!

      1. 1

        This comment was deleted 10 months ago.

        1. 2

          Effectively SurrealDB is the database and API layer in one, like Firebase, but built as a database not a data/api layer, and with powerful querying abilities for advanced analysis of data.

          Put simply, SurrealDB speeds up the development time of building an app/platform, and also massively reduces the maintenance time. So instead of focusing on the backend (with the database layer and the api layer), you only have to focus on your frontend app/platform.

          Does that make sense?

          1. 1

            This comment was deleted 10 months ago.

            1. 2

              Hi @ShaneBellone that's really helpful, and case studies and examples are something we definitely do need to add to the site! We have also planned to do some videos showing how to get started easily with SurrealDB, and what benefits you can get with it.

              To answer your question fully, I would need to know what kind of things you are building on top of MongoDB, so forgive me if any of this answer is irrelevant / wrong!

              I think the first advantage over a database like MongoDB, is the ability to build frontend applications (React, Ember.js Angular.js, Vue.js, ....) directly against the database, without having to manage the API layer or security or permissions. That's obviously a major difference to a database like MongoDB. With SurrealDB you can use SurrealQL directly from within Google Chrome (or another browser) over WebSockets, and we have plans for GraphQL too, coming soon!

              The second advantage to using SurrealDB over MongoDB is the ability to operate in schemafull or schemaless mode. So you could choose to use it just like MongoDB, or you can choose to use it like MySQL/PostgreSQL, but with complex document nested fields, arrays, and objects. You can specify advanced constraints on any field...

              DEFINE FIELD country.code ON user TYPE string ASSERT $value != NONE AND $value = /[A-Z]{3}/;

              UPDATE user:tobie SET country.code = 'United Kingdom'; // Will error

              The third advantage is the expressiveness of the query language. There are record links (direct links to foreign records which you can traverse as if the record was embedded in the document), and there is full graph functionality (person->connected_to->person). It's really easy to work with inter-connected data sets in SurrealDB, compared to the MongoDB query language...

              SELECT ->purchased->product<-purchased<-person->(purchased WHERE created_at > time::now() - 3w)->product FROM person:tobie;

              There are loads of other differences / features / benefits. The best page to see these, at a glance, is the features page.

              1. 3

                This comment was deleted 10 months ago.

                1. 2

                  No problem @ShaneBellone. Conveying the features/benefits/functionality of the product, and producing the documentation is almost harder than building the database itself!! Any comments, feedback, and questions are always super helpful! Do feel free to ask any other questions as it helps us with learning how to convey the SurrealDB benefits!

                  1. 1

                    This comment was deleted 10 months ago.

Trending on Indie Hackers
How I grew a side project to 100k Unique Visitors in 7 days with 0 audience 48 comments Competing with Product Hunt: a month later 33 comments Why do you hate marketing? 28 comments $15k revenues in <4 months as a solopreneur 14 comments Use Your Product 13 comments How I Launched FrontendEase 13 comments