‘Uncategorized’
Technical Writing and its ‘Hierarchy of Needs’
Wednesday, February 2nd, 2022
Technical writing is hard to do well and it’s also a bit different from other types of writing. While good technical writing has no strict definition I do think there is a kind of ‘hierarchy of needs’ that defines it. I’m not sure this is complete or perfect but I find categorizing to be useful.
L1 – Writing Clearly
The author writes in a way that accurately represents the information they want to convey. Sentences have a clear purpose. The structure of the text flows from point to point.
L2 – Explaining Well (Logos in rhetorical theory)
The author breaks their argument down into logical blocks that build on one another to make complex ideas easier to understand. When done well, this almost always involves (a) short, inline examples to ground abstract ideas and (b) a concise and logical flow through the argument which does not repeat other than for grammatical effect or flip-flop between points.
L3 – Style
The author uses different turns of phrase, switches in person, different grammatical structures, humor, etc. to make their writing more interesting to read. Good style keeps the reader engaged. You know it when you see it as the ideas flow more easily into your mind. Really good style even evokes an emotion of its own. By contrast, an author can write clearly and explain well, but in a way that feels monotonous or even boring.
L4 – Evoking Emotion (Pathos in rhetorical theory)
I think this is the most advanced and also the most powerful particularly where it inspires the reader to take action based on your words through an emotional argument. To take an example, Martin Kleppmann’s turning the database inside out inspired a whole generation of software engineers to rethink how they build systems. Tim or Kris’ humor works in a different but equally effective way. Other appeals include establishing a connection with the reader, grounding in a subculture that the author and reader belong to, establishing credibility (ethos), highlighting where they are missing out on (FOMO), influencing through knowing and opinionated command of the content. There are many more.
The use of pathos (sadly) doesn’t always imply logos, often there are logical fallacies used even in technical writing. Writing is so much more powerful if both are used together.
Related: livingston county, ny car accident, russell poole a cop we should insist on, jobs in salem oregon craigslist, petrina johnson and robert crisp still together, is aunt capitalized in spanish, log cabin kits reno nv, virginia covid sick leave 2021, ette place, can i fly with a cough coronavirus, mobile homes for rent in claremore, ok, avianca covid test requirements el salvador, michelle branca lee, population of aberystwyth, evergreen job requisition successfactors, terraria seed for jungle temple,Related: lamar hunt family tree, who is professor waldman in frankenstein, modern farmhouse virtual tour matterport, christus health employee benefits, daniel m melber seattle, dover banger racing fixtures 2022, is ukee washington in quarantine, roadtrek zion used for sale, how to clean old military uniforms, 2016 forest river sandpiper destination, lymphatic massage swindon, rehome french bulldog, how many copies has minecraft sold 2022, communion wine sainsbury’s, jen herro age,Related: bar louie voodoo sauce recipe, your tickets finished uploading and are now processing seatgeek, lord beaverbrook net worth, best photography locations near worcester, ma, lelit elizabeth v3 vs profitec pro 300, binghamton housing authority executive director, jasper county inmate roster, how to bleed air from ice maker line, gary edwards obituary, luke bryan farm tour 2022, upnorthlive car accident today, referee march madness, wslregisterdistribution failed with error 0xc0000005, dorset police helicopter tracker, bustle horoscope 2022,Related: cbp training academy housing, grants for barn restoration, tendaji lathan mother, fell compassionate analogy, how old was sandra bullock in hope floats, ejmr finance rumor, medders funeral home crossett, arkansas obituaries, tricare east corrected claims, how to change reporting lines in powerpoint org chart, cheap apartments winston salem, miami beach 1990s nightclubs, healy foundation new mexico, alone show worst injuries, cypress, tx weather monthly, dallas piano competition,Related: cyclace exercise bike manual, what is smart card pairing on my mac, octopus benefits testosterone, average mile time for college soccer players, clearwater, florida obituaries 2022, green goblin laugh sound effect, wrat 4 scoring guide, bobby flay restaurants los angeles, christian spiritual retreats near me, are pending charges included in total balance amex, sonic boom today california, keplinger funeral home obituaries, propane house explosion, owl attack human injuries, where is george from what the hales,Related: how to turn off water blur in the forest, homes for rent by owner vineland, nj, findlay city schools staff directory, school district of philadelphia climate manager salary, 3 week itinerary italy and switzerland, the brand closet employee login kate spade, madison county, va obituaries, relocation assistance jp morgan, columbus zoo arctic fox anana death, charles joshua powell, paloma tree fataar, mass state retirement chart group 2, flamingos restaurant menu, is timothy busfield in a wheelchair, houses for rent in kissimmee under $1200,Related: what happened to larry hinson, kleenheat contact number, elemental grind game code, tauranga hospital orthopaedics, shippensburg university basketball, former komo news reporters, command style coaching pros and cons, barrett jackson auction, haycombe crematorium schedule, tuscany wedding packages, 765 rockbridge rd, montecito, ca 93108, guns n’ roses tribute band massachusetts, birchen clough scramble, beaver dam raceway tickets, zivilisatorisches hexagon beispiel irak,Related: barry soper ex wife, con que puedo sustituir el nopal en una dieta, goffs school teachers, noodler’s ink controversy, marriott staff directory, civil standby colorado, waterfront homes for sale yamba, nsw, quartz crystal cave oregon, thank you to church family after funeral, black funeral homes in fort pierce, fl, 54 bus on sunday, iterate through list of dictionaries python, how does hsa work with child support, self mutilation waiver air force, splinter removal kit walgreens,Related: usps background check red flag, where to find geodes in south carolina, argyle baseball roster 2021, alternative to gruyere cheese in dauphinoise, how to make multichrome eyeshadow, the lighthouse mermaid scene, bob uecker siblings, is army of two 40th day backwards compatible on xbox one, skinwalker deer video, gila county jail payson, az mugshots, , firefighter powerpoint presentations, casey johnson obituary, how to fix peeling shirt printing, cabrini cardiologists,Related: how to allocate more ram to sims 4, optimo cigars expiration date, how did antoinette chanel die, calories in 1 tbsp brown sugar, joey rourke cause of death, minecraft but you can combine any items mod, meijer cake catalog, amado carrillo fuentes’ death, spencer reid maeve, is mrcp recognised in canada, mitsuhiko kanekatsu’s bww vin decoder, le desenvoutement dure combien de temps, platform technologies syllabus ched, aaron jones nicknames, iron ii chromate formula,Related: passaic high school staff directory, arizona modern furniture, cameron rachel hamill, ahimelech and abimelech, underrail quest order, worcester arrests yesterday, ovary pain when walking, boronia beach penguins, small wedding venues virginia, fm 590pp non dot urine labcorp, meghan markle mean to charlotte, etsy removable wall murals, john bolling descendants, central bedfordshire tidy tip booking, spotify premium family invite,Related: big john studd vietnam, what replaced redken diamond oil, cooper creek campground georgia, taupo death notices 2021, mecklenburg county candidates 2022, mayfield middle school bell schedule, anthurium jenmanii variegated, where does lolo jones live now, farruko health problems, alamo heights football roster, shortened descriptor example, what gyms accept issa certification, st cuthbert’s school, newcastle staff list, alameda county police scanner, who died on shameless in real life,Related: john lippoth obituary, shopping in bay st louis, jet2 advert 2020 actress, does evening primrose oil make your breasts bigger, aws glue api example, wheatgrass histamine intolerance, jetstar vaccination policy within australia, all of the following are presidential roles except, reasons for failure of moon treaty, which city in new zealand has the highest crime rate, medical surgical assessment exam quizlet, ponchatoula, louisiana murders, brands like blackbough swim, , coast g25 flashlight manual,Related: how to take apart a kohler kitchen faucet, was steven seagal a navy seal, native plants for erosion control, savage 93 22 mag magazine problems, jerome ghost tour groupon, ford edge throttle body recall, cellular network not available for voice calls moto g7, operation odessa where are they now, sea of thieves can’t change resolution, 1950 craftsman tool catalog, 2010 kubota rtv 900 specs, roseburg oregon tv channels, mastidane puppies for sale near me, tony robinson nfl philadelphia, sisters of st mary peekskill, ny,Related: new construction homes under $250k near me, texas southern university sororities and fraternities, tavistock police news, holly pollard net worth, ohio liquor license search, what is an example of mutualism in the tropical rainforest, watkins mill youth detention center, funny baseball awards, grace and frankie when did mallory have twins, martha paiz fogerty, what does cps look for in a home study, gibson l3 for sale, willow pump blinking red while charging, roane county recent obituaries, difference between scabies and ringworm,Related: sample motion for temporary orders massachusetts, eybl team tryouts 2022, is left axis deviation ecg dangerous, 30 day weather forecast for montana, cuanto tarda en secar la tinta para madera, remove speed limiter on mobility scooter, grant parish school board pay scale, quien es el esposo de coco march, security jobs paying $30 an hour, dreamline shower base drain size, pgh laparoscopic surgery cost, mitchell goldhar wife, union county section 8 plainfield, nj, jackson high school basketball schedule 2021, south africa surnames,Related: wasserstein private equity, walgreens nationals logo lawsuit, triton boats apparel, how to sneak your phone in a jail visit, albertsons software engineer salary near berlin, aziende biomediche svizzera, how did richard karn lose weight, danny kelly wife, who is the mom in the liberty mutual nostalgia commercial, where does martina navratilova live in miami, normal cranial vault asymmetry index, nextamz wireless thermometer manual, 2019 california green building standards code dwg, can ‘t smile after septoplasty, katie lange age,Related: men’s minimal coverage swimwear, how to defer a ticket in king county, is the bubble room haunted, levi strauss foundation executive director salary, joe zolper t shirts, bladen county crime, laporte county assessor property search, elliot hospital cafeteria menu, whitby to scarborough boat trip, usc applied data science faculty, what does it mean when a man calls you boss, 3 types of licence issued under the licensing act 2003, who is letitia james parents, joanna and shariece clark update 2021, wa lottery app says please see lottery,Related: conservative razor companies, the parsons family murders, fishtail palm poisonous to dogs, do you need a license to breed snakes in california, 12 gauge round ball, whodini vh1 hip hop honors performance, ke huy quan wife, brandon davis obituary, mike and kelly bowling divorce, police chase in plano tx today, kyker funeral home harriman, tn obituaries, luscombe 8a bush plane, return enchantment from graveyard standard, bend, oregon mugshots 2021, saintsations auditions 2021,Related: pets at home photo competition 2021, southern district of texas, list of funerals at three counties crematorium, zero emission vehicle companies, eml newcastle address, gila county mugshots, steve reevis death cause, jupiter pizza menu nutrition, anchorage airport live camera, ffxiv too close to home report to central shroud, ansa keyboard shortcuts, watertown building department, nys dmv registration renewal form, zaandam refurbishment 2020, how old is kelly austin,Related: 2022 nfl mock draft simulator, how long does air duster stay in your system, osha does not approve individual states true or false, nebraska missing persons, lake of the woods real estate oregon, western intercollegiate golf leaderboard, jefferson hospital patient information, housing authorities absorbing vouchers, terri halperin married, slimline tumble dryer 45cm wide currys, what is the name of c3n4 compound name, what is the first step of the spider method, which three statements are accurate about debug logs, where did muhammad ali live before he died, dysautonomia covid vaccine reaction,Related: top 10 dangerous caste in pakistan, panda express calorie calculator, everlast gym opening times, shooting in rialto today, can i take shilajit with coffee, short badass military quotes, 3 grand trines in natal chart, pennsbury high school football, accidentally cooked plastic with food, the ancient and noble house of black revolution fanfiction, taurus man fantasy gemini woman, sturm funeral home, sleepy eye obituaries, trevor siemian career earnings, east alton police blotter, ashley county warrant list,Related: khnadya skye nelson age, mary elizabeth mcdonough, cheryl araujo daughters where are they now, ako prejavuje lasku vodnar, university of miami volleyball coach email, why do monkeys reject their babies, superintendent humble isd, when do orioles leave michigan, scottish terrier bite force psi, unsolved murders in fayetteville, nc, dirk diggler prosthetic picture, skytech m1000 mouse software, bishopric youth discussion handbook, current road conditions boulia to birdsville, filing a false order of protection illinois,Related: general jack keane girlfriend, who killed tyler in a dark place spoiler, how much bigger is earth than mars in km, texas franchise tax public information report 2022, asgore fight simulator github, gm financial late payment removal, contact alo yoga customer service, west ashley accident today, tufts baseball recruiting questionnaire, tatum coffey wedding, nen ability generator, gannon shepherd wife, sydney swans academy players 2021, who was ogden stiers partner, south dakota gun laws for non residents,Related: new york obituaries archives, bungalows for sale in waltham abbey, commodification of hawaiian culture, dallas construction projects 2022, is nico de boinville married, who is betty klimenko husband, thigh tattoo hurts to walk, otto ohlendorf descendants, menards coming to gaylord, mi, jamie oliver 123 traybake, how did alejandro family die in sicario, purdue university dorms, starting specific gravity for moonshine mash, explain col using at least three descriptors, grants for private practice counseling,Related: am i attractive to guys quiz buzzfeed, how long to walk around port lympne zoo, lancaster county school district salary schedule, adjectives to describe ray bradbury’s life, tesla model 3 grinding noise, muscle twitching all over body at rest forum, are rachel and mitchell moranis twins, johnny depp horoscope, screen mirroring windows 11 to tv, is harold perrineau really in a wheelchair, top hernia surgeons long island, mdot executive director, barrow neurosurgery fellows, world’s strictest parents tamsin update, nigel clough wife,Related: tiny times 4 ending explained, nothing to declare australia nadia, expert mode terraria boss drops, fran lebowitz ellen lebowitz, muddy paws rescue omaha, gillespie county election results 2022, ralph capone jr, harry and meghan snubbed by spotify, john hammergren family, celebration of life venues portland oregon, american manchester terrier club rescue, did kelly preston have chemotherapy for her cancer, why did they stop selling jolly ranchers in the uk, juegos para mayores de 18, don gordon wife,Related: iowa 1st congressional district 2022, st francis hospital cafeteria menu, chrysler dealer code lookup, what happens when you win a microsoft sweepstakes, warzone unlock all tool discord, peterson and williams funeral home obituaries, brogo dam canoe hire, halifax courier archive obituaries, does catherine oxenberg have a royal title, navarre press arrests, isabelle townsend today, warble home remedy, what to wear to a wardruna concert, what to say when someone says trust me, xkw1 switch hack 2021,Related: things to do between grand canyon and moab, royal burger bernie mac, kennesaw mountain high school news, palm harbor homes class action lawsuit, senior open qualifying 2022 entry form, pickleball courts bellevue, franklin county busted, fedex ground rates per pound, yousif tlaib, how tall was clint walker’s twin sister lucy, curtis pilot polo net worth, ohio college of podiatric medicine ranking, east hampton, ct woman found dead, christus health employee benefits, how much weight can a 2×3 support horizontally,Related: shelby 5101 trailer vin location, illegal eagles christian, lynne thigpen obituary, what does tom hagen say to vincenzo pentangeli, houses for rent statham, ga, quien fue azeneth, tree preservation order map cardiff, crooked lake bc cabin for sale, nexrad level 3 data feed, rick stacy morning show today, mary mcgowan attorney virginia, roan mountain state park trail map, jasper county ga obituaries, zakros ancient tablet, how to use debit card before it arrives,Related: megan rapinoe fan mail address, ashtabula county sheriff reports, shooting in cookeville, tn today, disadvantages of chemical synapses, haysville, ks warrant search, eccles tram stop to aj bell stadium, moderation management pdf, what happens if you eat expired pez candy, coliban potato substitute, was johnny cash museum damaged in explosion, which camp buddy character are you quiz, north coast wine transport, quadratic graph calculator with points, lollapalooza 2022 lineup, acacia acuminata queensland,Related: omma testing requirements, benchmade infidel upgrades, tesco policies and procedures, penn state 1987 football roster, error during websocket handshake: unexpected response code: 404, pebble shore lake montana map, plastic surgeon st vincent’s private hospital, st james high school football record, lind family funeral home obituaries, wbbj crime stoppers, glamrock freddy voice, lipscomb university lectureship, watertown building department, how much health does templar have destiny 2, choking on liquids after thyroidectomy,Related: oldies concerts 2022 california, does one love mean i love you, pickering creek reservoir boating, absolute roughness of stainless steel, joanne froggatt downton abbey, ace model 1000 garbage disposal installation manual, city of tempe setback requirements, how many times can 8 go into 2, kirk cousins house, past mayors of danbury, ct, apartment comparison spreadsheet google sheets, what size easel do i need for a 16×20 canvas, dell s2721qs calibration settings, new york to miami sleeper train, does heinz simply ketchup taste different,Related: boshamer stadium rules, lenny breau daughter, quarter horses for sale in massachusetts, south ribble council contact number, aics thumbhole upgrade, can an ovarian cyst cause leukocytes in urine, miami airport to eden roc hotel, travelodge saver rate cancellation, new balance commercial baseball player, in texas party politics today quizlet, north carolina symphony musicians, how far can a bobcat jump horizontally, nfl communications staff directory, carvelli restaurant group, accident on 223 adrian, mi today,Designing Event Driven Systems – Summary of Arguments
Thursday, October 4th, 2018
This post provides a terse summary of the high-level arguments addressed in my book.
Why Change is Needed
Technology has changed:
- Partitioned/Replayable logs provide previously unattainable levels of throughput (up to Terabit/s), storage (up to PB) and high availability.
- Stateful Stream Processors include a rich suite of utilities for handling Streams, Tables, Joins, Buffering of late events (important in asynchronous communication), state management. These tools interface directly with business logic. Transactions tie streams and state together efficiently.
- Kafka Streams and KSQL are DSLs which can be run as standalone clusters, or embedded into applications and services directly. The latter approach makes streaming an API, interfacing inbound and outbound streams directly into your code.
Businesses need asynchronicity:
- Businesses are a collection of people, teams and departments performing a wide range of functions, backed by technology. Teams need to work asynchronously with respect to one another to be efficient.
- Many business processes are inherently asynchronous, for example shipping a parcel from a warehouse to a user’s door.
- A business may start as a website, where the front end makes synchronous calls to backend services, but as it grows the web of synchronous calls tightly couple services together at runtime. Event-based methods reverse this, decoupling systems in time and allowing them to evolve independently of one another.
A message broker has notable benefits:
- It flips control of routing, so a sender does not know who receives a message, and there may be many different receivers (pub/sub). This makes the system pluggable, as the producer is decoupled from the potentially many consumers.
- Load and scalability become a concern of the broker, not the source system.
- There is no requirement for backpressure. The receiver defines their own flow control.
Systems still require Request Response
- Whilst many systems are built entirely-event driven, request-response protocols remain the best choice for many use cases. The rule of thumb is: use request-response for intra-system communication particularly queries or lookups (customers, shopping carts, DNS), use events for state changes and inter-system communication (changes to business facts that are needed beyond the scope of the originating system).
Data-on-the-outside is different:
- In service-based ecosystems the data that services share is very different to the data they keep inside their service boundary. Outside data is harder to change, but it has more value in a holistic sense.
- The events services share form a journal, or ‘Shared Narrative’, describing exactly how your business evolved over time.
Databases aren’t well shared:
- Databases have rich interfaces that couple them tightly with the programs that use them. This makes them useful tools for data manipulation and storage, but poor tools for data integration.
- Shared databases form a bottleneck (performance, operability, storage etc.).
Data Services are still “databases”:
- A database wrapped in a service interface still suffers from many of the issues seen with shared databases (The Integration Database Antipattern). Either it provides all the functionality you need (becoming a homegrown database) or it provides a mechanism for extracting that data and moving it (becoming a homegrown replayable log).
Data movement is inevitable as ecosystems grow.
- The core datasets of any large business end up being distributed to the majority of applications.
- Messaging moves data from a tightly coupled place (the originating service) to a loosely coupled place (the service that is using the data). Because this gives teams more freedom (operationally, data enrichment, processing), it tends to be where they eventually end up.
Why Event Streaming
Events should be 1st Class Entities:
- Events are two things: (a) a notification and (b) a state transfer. The former leads to stateless architectures, the latter to stateful architectures. Both are useful.
- Events become a Shared Narrative describing the evolution of the business over time: When used with a replayable log, service interactions create a journal that describes everything a business does, one event at a time. This journal is useful for audit, replay (event sourcing) and debugging inter-service issues.
- Event-Driven Architectures move data to wherever it is needed: Traditional services are about isolating functionality that can be called upon and reused. Event-Driven architectures are about moving data to code, be it a different process, geography, disconnected device etc. Companies need both. The larger and more complex a system gets, the more it needs to replicate state.
Messaging is the most decoupled form of communication:
- Coupling relates to a combination of (a) data, (b) function and (c) operability
- Businesses have core datasets: these provide a base level of unavoidable coupling.
- Messaging moves this data from a highly coupled source to a loosely coupled destination which gives destination services control.
A Replayable Log turns ‘Ephemeral Messaging’ into ‘Messaging that Remembers’:
- Replayable logs can hold large, “Canonical” datasets where anyone can access them.
- You don’t ‘query’ a log in the traditional sense. You extract the data and create a view, in a cache or database of your own, or you process it in flight. The replayable log provides a central reference. This pattern gives each service the “slack” they need to iterate and change, as well as fitting the ‘derived view’ to the problem they need to solve.
Replayable Logs work better at keeping datasets in sync across a company:
- Data that is copied around a company can be hard to keep in sync. The different copies have a tendency to slowly diverge over time. Use of messaging in industry has highlighted this.
- If messaging ‘remembers’, it’s easier to stay in sync. The back-catalogue of data—the source of truth–is readily available.
- Streaming encourages derived views to be frequently re-derived. This keeps them close to the data in the log.
Replayable logs lead to Polyglot Views:
- There is no one-size-fits-all in data technology.
- Logs let you have many different data technologies, or data representations, sourced from the same place.
In Event-Driven Systems the Data Layer isn’t static
- In traditional applications the data layer is a database that is queried. In event-driven systems the data layer is a stream processor that prepares and coalesces data into a single event stream for ingest by a service or function.
- KSQL can be used as a data preparation layer that sits apart from the business functionality. KStreams can be used to embed the same functionality into a service.
- The streaming approach removes shared state (for example a database shared by different processes) allowing systems to scale without contention.
The ‘Database Inside Out’ analogy is useful when applied at cross-team or company scales:
- A streaming system can be thought of as a database turned inside out. A commit log and a a set of materialized views, caches and indexes created in different datastores or in the streaming system itself. This leads to two benefits.
- Data locality is used to increase performance: data is streamed to where it is needed, in a different application, a different geography, a different platform, etc.
- Data locality is used to increase autonomy: Each view can be controlled independently of the central log.
- At company scales this pattern works well because it carefully balances the need to centralize data (to keep it accurate), with the need to decentralise data access (to keep the organisation moving).
Streaming is a State of Mind:
- Databases, Request-response protocols and imperative programming lead us to think in blocking calls and command and control structures. Thinking of a business solely in this way is flawed.
- The streaming mindset starts by asking “what happens in the real world?” and “how does the real world evolve in time?” The business process is then modelled as a set of continuously computing functions driven by these real-world events.
- Request-response is about displaying information to users. Batch processing is about offline reporting. Streaming is about everything that happens in between.
The Streaming Way:
- Broadcast events
- Cache shared datasets in the log and make them discoverable.
- Let users manipulate event streams directly (e.g., with a streaming engine like KSQL)
- Drive simple microservices or FaaS, or create use-case-specific views in a database of your choice
The various points above lead to a set of broader principles that summarise the properties we expect in this type of system:
The WIRED Principles
Windowed: Reason accurately about an asynchronous world.
Immutable: Build on a replayable narrative of events.
Reactive: Be asynchronous, elastic & responsive.
Evolutionary: Decouple. Be pluggable. Use canonical event streams.
Data-Enabled: Move data to services and keep it in sync.
Slides from Craft Meetup
Wednesday, May 9th, 2018
The slides for the Craft Meetup can be found here.
Building Event Driven Services with Kafka Streams (Kafka Summit Edition)
Monday, April 23rd, 2018
The Kafka Summit version of this talk is more practical and includes code examples which walk though how to build a streaming application with Kafka Streams.
Slides from Strata Software Architecture: The Data Dichotomy – Rethinking data and services with streams
Wednesday, April 5th, 2017
QCon 2017: The Power of the Log
Wednesday, March 8th, 2017
VIDEO HERE
This talk is about the beauty of sequential access and append only data structures. We’ll do this in the context of a little known paper entitled “Log Structured Merge Trees”. LSM describes a surprisingly counterintuitive approach to storing and accessing data in a sequential fashion. It came to prominence in Google’s Big Table paper and today, the use of Logs, LSM and append only data structures drive many of the world’s most influential storage systems: Cassandra, HBase, RocksDB, Kafka and more. Finally we’ll look at how the beauty of sequential access goes beyond database internals, right through to how applications communicate, share data and scale.
Streaming, Databases & Distributed Systems – Bridging the Divide
Wednesday, November 23rd, 2016
This talk introduces Stateful Stream Processing and makes a case for SSP as a general approach to data computation in distributed environments.
Slides, alone, can be found here:
Slides from Codemesh & BigDataLdn
Friday, November 4th, 2016
Slides for JAX London
Wednesday, October 12th, 2016
Same title, but different content to the QCon one.
Slides from QCon: Microservices for a Streaming World
Monday, March 7th, 2016
Full talk can be found HERE
Abstract for Code Mesh 2015
Monday, July 20th, 2015
Contemporary Approaches to Data at Scale (tbc)
We use a host of tricks these days for handling data at scale. Disk structures are tuned to specific workloads. Streams are used to create continuous pipelines of processing. Hardware offers incredible diversity in terms of latency and throughput.
The tools available: Cassandra, Postgres, Hadoop, Kafka, Hazelcast, Storm etc all come with tradeoffs unique to themselves. We’ll look at these as individual elements. We’ll also look at compositions that leverage these individual sweet spots to create more powerful, holistic platforms.
Abstracts for Øredev 2015
Thursday, July 9th, 2015
The Future of Data Technology (6th Nov 15.40)
No longer does one-size-fit-all when it comes to data technology. At least not for many of today’s use cases. Will this ever change? Will we continue to diversify? Will we go full circle? Certainly ours is an industry in flux. NoSQL, Big Data and stream technology, containerisation, commodity PCIe storage, non-volatile memory and a host of other forces will shape the data technologies of the future.
In this talk will make a case for what the future may look like, what challenges we’ll encounter and how it will likely change the applications we build.
Elements of Scale: Composing And Scaling Data Platforms (5th Nov 14.20)
Today there are a host of data-centric challenges that need more than a single technology to solve. Data platforms step in, blending different technologies to solve a common goal.
But to compose such platforms we need an understanding of the tradeoffs of each constituent part: their sweet spots, how they complement one another and what sacrifices they make in return.
This talk is really a grand tour of these evolutionary forces. We’ll cover a lot of ground, building up from disk formats right through to fully distributed, streaming and batch driven architectures. In the end we should see how these various pieces come together to form a pleasant and useful whole.
Abstract for JAX London 2015
Thursday, July 9th, 2015
Intuitions for Scaling Data-Centric Architectures (14th Oct 11.20)
This talk will examine the various intuitions and trade-offs needed to scale a data-centric application or architecture. Building from the fundamentals of data locality, immutability and parallelism, attendees will gain a sense for how fully blown architectures can be sewn together. The result: a balance of real-time storage, streaming and analytics that plays to the relative strengths of different component parts.