This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git.
from 48865e0 Merge pull request #6653 [BEAM-5702] Special case zero and one element Creates. add f9b0d5c #! Init Euphoria 0.5.0 add 30645cf #! minor code cleanup add f460a32 #! Fix dead lock when executing many tasks add 19a177b #! Split partition description from its reader add 3b4301e #! Call Reader#close upon end-of-stream add 9eec902 #! Initial draft of KafkaStreamSource add 7f389b2 #! Correct typo in classname add 12dfbfe #! Upgrade PrintStreamSink to a general purpose utility add be18d10 #! Revise comments in example code add 0ebd39f #! Allow kafka configuration through properties resource add 48a0c9e #! Fix compilation error in IDEA add 441ceda #! Propose readable variable/parameter name add 9241efd #! Hadoop input formats adapter add 0df1bc0 #! Hadoop input format - clone Writable instances between RecordReader#nextKeyValue calls add 7bfe5f7 #! Unify input/output creation add 64893eb #! Provide easy access to nested configuration settings add 0ed8e60 #! Make dataset aware of its consumers add 8e3c625 #! added basic test for retrieving dataset's consumers add 94a0094 #! fix typo in comment add 907442f #! Make kafka source configurable through settings add ecd7f55 #! Make StdoutSink#dump-partition-id parameter optional again add 66da90a #! DataSink API polishing add 6513da4 #! Do not close stdout through StdoutSink add 2308fb9 #! Log swallowed exceptions add bf51c4b #! InMemExecutor - fix potential concurrency issue add 786af3c #! InMemExecutor - rollback all sinks when any of the tasks fails add 3c94462 #! Add inmemory filesystem skeleton add 12425ef #! Apply InMemFileSystem as SoR in test add c313c27 #! Drop obsolete/unused method add ae01d8c #! Rework iteration towards java8 streams add 302696b #! Remove unused imports add 613e870 #! Working join operator add 4b42d50 #! git pull --rebase origin master add 8f7aa0b #! minor polishing and removing of warnings add b99b765 #! polishing after code review add 9f7aa47 #! correct calling of constructor of ReduceStateByKey from GroupReduceByKey add 34f4284 #! pair moved package add 31d9e77 #! Hadoop output format adapter add 52c3a0a #! clone datasets to all consumers in inmem executor add b8ffe90 #! KafkaStreamSource -> KafkaSource add 9ab4989 #! Initial draft of a non-tx-aware kafka-sink add c1bba39 #! Correct Pair class reference add 0650f47 #! comment typo add 20acd1e #! first part of code review changes add b10d828 #! core of realtime stats add 6ad0a65 #! Ensure writing out EOS token add b1276b4 #! DRY on computing sum add 8873830 #! implemented remarks from code review add 321ecc8 #! Get reasonable logging output during tests add c708ccf #! InMemFileSystem.Factory -> InMemFileSystem.SourceFactory add 0113723 #! Sink into the InMemFileSystem for unit testing purposes add 8de5c08 #! Drop CompositeKey#equals implementation add f152502 #! Avoid eye-bleeding add 9ab0900 #! Avoid doubled output add e1b3e9f #! added tests for flatmap, repartition and union, slightly cleaned code in inmem add 538c42c #! more correct commit of ListDataSink's Writer add 7b62cb0 #! get rid of batch code in client API add f75b99d #! changes after review add 3cbfb89 #! Staless Windowing (Windowing.Count working; others commented out) add 729227d #! Attempt at merging add 3857bd9 #! Fix state merging add 186551b #! Drop Window#add/getState add adfafea #! Shorten test code add b91d002 #! Aligned count window add 9f58185 #! Add Join implementation add 0124d19 #! Finish up join implementation add aa76e98 #! fixed test for sort in reducestatebykey add 635f04b #! Various styling issue; thanks to review feedback add 6c04bb8 #! Add more debugging support through stdout-sink add c0c640b #! Correct output format add 4de9f2e #! added tests for reducestatebykey - non aggregating and unaligned windows add 6c42127 #! changes after review add 3db6d4e #! Provide more debugging info through stdout-sink add bd9b25d #! Add (failing) proof-of-concept test add 2af3656 #! Allowing windowing by item specific event-item add 97e1234 #! Add VoidSink for debugging purposes add 8af649a #! Prefer UnaryFunction over EventTimeFn add 43b9d29 #! Prevent timer thread to die add ea6d25e #! Prepare window storage add 1f6cd09 #! Faster retrieval of active windows add 1c1b0d4 #! added tests for inmem, added sliding window add 016544d #! fixed copying inputs in multi consumer pipeline add 4b3d048 #! SlidingWindow constructor made private add 9e9f68c #! Fix import add 63abc3d #! Fix type reference add 2d4b850 #! changed triggering to allow event time add af4f7c5 #! added watermark triggering with basic test add 497a35e #! fixed tests, code polishing add 7566a03 #! simplified tests, added element discarding on closed windows add 4f2fdbd #! better handling of bounded inputs with triggering add 0d18e2c #! API - refactor operator builders + unit tests add c745d91 #! Propage window labels to operator output add decd53b #! Drop needles consistency methods add b1d2aa4 #! Correct variable name add bc10c94 #! Attached/DefaultWindowing emission add ce2a886 #! Add test add 58fbe2d #! EndOfPane -> EndOfWindow add 55ae246 #! Put State#collector behind a getter add 72d83a4 #! Move EndOfWindow skiping directly into InMemExecutor add f5fa99a #! Drop DatumCleanupSink add 8f618e8 #! Move InMemExecutor into a dedicated package add 8021895 #! Drop obsolete PersistableDataset add 6706ced #! Tighten test condition; provide explanation add ca0849d #! Drop prepared logging in test add 6a2637b #! Tune test add 05a8741 #! Forward end-of-window only when arriving on all input partitions add 70bf7b6 #! Broadcast EoW to every output partition add 168f17a #! Drop forgotten debug message add b6fc082 #! Fix emitting EoW too frequently add a264322 #! Do not synchronize when invoking Windowing#assignWindows/#updateTriggering add 345e176 #! Prepare for EoW tracking add 0022817 #! Avoid broadcasting too many times add 4706d5b #! Drop debugging log messages add a3084c1 #! Fix trigger rescheduling in WatermarkTriggering add 2af6762 #! Style changes add d80d8f7 #! Resolve merging of obsolete windows add 60f1a6d #! added tests applicable on all executors add 39fd87c #! removed operator Mapwindow add 3a84c06 #! added simple wordcloud application add 6688c97 #! Use attached windowing in Stats add d6534c1 #! Enable watermarking in Stats again add e580bfb #! compilability after rebase add 81d9294 #! Attempt at nailing down random test failure add 2d3cefe #! Allow unified access to .output() add eb2598b #! Fluent flow builder (proof-of-concept) add 71e4a9f #! Fluent example add 73f7c08 #! Move fluent api to a submodule add 3b57637 #! Rename OutputProvider to OutputBuilder add c48f662 #! Another attempt at nailing down random test failure add c284805 #! Attempt at fixing flipping test add 769446e #! polishing after review add 5581e52 #! Update maven repo pointers add b8b59a3 #! Fix maven-project-info-reports-plugin version add d3495aa #! early triggering of window (replaces aggregating windows) add b93c13c #! Thread-safe access to #activeTasks add ba9ede9 #! Javadoc improvement and minor bug fixes after review add 0773952 #! Windowing API requires fixed order of builder methods add 297ba2b #! Allow reading kafka a whole topic as one partition (for easier debugging purposes) add eb4e595 #! Utilize eventTime and earlyTriggering in Stats (WatermarkWindowing disabled) add e8cf2bd #! Revert back to time-sliding add abdc4fb #! Allowing limiting the RSBK key space add da88f9a #! Drop verbose debug message add e282016 #! Hide implementation details and make public input based on Triples add 8aa4dc7 #! Fix evaluating optional setting add de51518 #! Introduce session windowing add 72863d7 #! Rename span -> createSpanned add 41b7143 #! Fix merging fully enclosed windows add bf9e3e9 #! Speed up ordering add 03d599f #! Test for session window labels add 3c8aa8a #! Fix time-sliding and session-windowing in combination with watermark-triggering add 2d70630 #! Allow consuming kafka topics from any earlier offset add ca6b0d0 #! Print debug to log instead of stderr add cafa1db #! Add failing proof-of-bug test add c6e95b1 #! Fix DAG traversal add fa8ab35 #! Prefer duration over millis in time based APIs add 682db92 #! Required TimeSliding step to be a duration in favor of millis add a572ec0 #! Make up session across searches and clicks add 2622383 #! Resuse Either#lift add 24e77b4 #! Make TopPerKey a proper operator add 15e6af9 #! Suppress unchecked warnings; Fix "grouped" flag in RBK add 3c752ba #! Fix indentation add fe213c2 #! Fix typo add e4d3fed #! Make CompositeKey public and reuse inside inmem executor add 9259979 #! Fix inmem executing R(S)BK on a grouped dataset add 489dbd3 #! Flink executor skeleton add 8effa40 #! fix compilation error add 5d4bb12 #! refactoring after review add 829d4d3 #! DataSource adapter for Flink executor add 70fa452 #! DataSink adapter for Flink executor add 88741c3 #! Support logging during flink tests add 69ab721 #! DataSink#rollback throws IOException add 5b0bf88 #! fix test add 38b6e36 #! ListDataSink stores data in static context add 96a6f8c #! Favor cz.seznam.euphoria:guava-shaded over com.google.guava:guava add 077dfdc #! Rebase on FlinkIO add 87ca8ca #! ListDataSink review add 9b5ce9a #! Flink implementation of ReduceByKey (no windowing) add 2e41081 #! ReduceByKey has isCombinable() method add c4b6434 #! ReduceByKey uses custom partitioner just in case it's different from HashPartitioner add 6f03b8a #! testapps - WordCount implementation add 92830a3 #! Remove explicit parallelism from OperatorTranslator interface add 58ee948 #! Comment add 078dbad #! Avoid casts for readability reasons add 5234600 #! [euphoria-flink] Dump execution plan prio to flow execution add 74132da #! Type-safe HadoopInputFormatAdapter. add 0b9d183 #! Type-safe HadoopOutputFormatAdapter. add e6c2625 #! Refactoring. add 01f2567 #! Make HadoopDataSource public. add e16db17 #! Add SequenceFile factories. add 6a96f0c #! Add proof-of-concept test add dc891c3 #! [euphoria-flink] Fix operator-input-dataset-mapping add 348f65e #! Add printing element type for debugging purposes add f1090c9 #! Avoid non-implemented windowing in proof-of-concept test add b67194d #! [euphoria-flink] Separation of streaming and batch executors add 944566f #! Rename items for clarity reasons add 11a610c #! Tighten method signature to expected operator type add ba959fa #! Javadoc add eaa29b6 #! Move #getExecutionEnvironment to parent class add 8eb5dda #! Remove garbage add 45ebae2 #! [euphoria-flink] batch input format adapter (without data locality) add db17fcc #! [euphoria-flink] batch output format adapter add a84e6a6 #! [euphoria-flink] batch ReduceByKey implementation add 14e8962 #! [euphoria-hadoop] Make HadoopDataSource serializable add 98ebe69 #! [euphoria-flink] Make batch-word-count work (on local-flink-environment) add 068279f #! [euphoria-flink] Add an upper bound to parallelism on operators add 5e27688 #! [euphoria-core] Conditional partitioning in fluent style add 8288cfb #! refactored window to windowcontext and windowid add 6a2be27 #! fixed test for time sliding window add dd89564 #! moved Datum to public API and changed its name add a3d28ff #! likvidace unchecked warningu add c579b9a #! refactored windowing stuff to separate package add 270f991 #! refactored windowing types to separate classes add 6f3299d #! unified AttachedWindowing with all other windowings add 4c2b840 #! unified ElementWindowing and Windowing add 04ae2de #! refactored data passing inside inmem pipelines, failing tests related to attached windowing add a76407e #! rebased on master add cdf9f79 #! inmem working on watermarks and regular attached windowing add 350f4c5 #! incorporated changes after review add e4da43a #! [euphoria-inmem] Emit watermarks with every input item add c193b49 #! Extract diff of jenda/16255-add-watermarking-to-inmem and master add 60f7b6e #! Make euphoria-flink compile (temporary work-around) add 373087f #! Increase read-timeout way over watermarking triggering in the inmem executor to stabilize test add 41924eb #! [euphoria-core] Drop obsolete field add c5b68b7 #! fix watermark triggering and associated tests add dc4db46 #! code polishing add 5a42ece #! removed unneeded synchronization add 09be59d #! [euphoria-flink] Partial progress on window-element envelopes add 4b6e783 #! [euphoria-flink] Handle window-element envelopes in Repartition operator add d1f3cf9 #! tuned generic windowing for use with sliding windows add a2d20b1 #! code formatting add ec52be0 #! [euphoria-flink] Handle window-element envelopes in batch mode add 098dd0c #! [euphoria-flink] Perform conversion to/from window-elements in sink/sources just as in streaming mode add e2a06a4 #! [euphoria-flink] Attempt fixing window leak add e8255e8 #! [euphoria-flink] Fix emiting label to windowed-pairs. add aa2f21e #! Split operator add 83aafb1 #! commit kafka consumer offsets add 6b4b7fd #! Split operator add 0a94203 #! post-review cleanup add 9a668dd #! refactoring add c90060f #! [euphoria-flink] Add context to error message add 28e7710 #! [euphoria-testapps] WordCloud Server/Client/Sink add f706930 #! [euphoria-flink] Initial sketch of JoinTranslator add 4ffe224 #! [euphoria-flink] Correctly emit Pair instances out of join add 584d550 #! add support for state backend to flink add 5da1726 #! [euphoria-flink] Handle windowed-element in join operator add ece01d8 #! tuned stats for inmem and new windowing add 6b97a79 #! stats tuned on inmem for event time, latecomers and correct distinct calculation add e15d983 #! fixed ReduceWindow operator add 7977ac4 #! simplified stats flow, fixed some bugs and minor refactored code and some coding style add 0f71979 #! [euphoria-flink] implementation of ReduceStateByKey for streaming API add 5dc7a0c #! [euphoria-flink] avoid "Invalid lambda deserialization" in ReduceByKey add b278dd3 #! [euphoria-flink] RSBK closes state after flush() add 0e8ac24 #! [euphoria-flink] implementation of ReduceStateByKey for batch API add d7e987c #! codestyle after review add 4584dfd3 #! [euphoria-flink] Fix NPE due to ReduceWindow add db7a29f #! validate that no two operators use the same sink add f2d2a91 #! [euphoria-flink] Fix class-cast-exception add 220e121 #! [euphoria-flink] Have StreamingWindowedElement flow through the streaming executor add f0e3054 #! [euphoria-flink] Drop support for processing time add 1290d88 #! [euphoria-flink] Add support for transfering window emission watermark add ef0872b #! [euphoria-flink] Add support for attached windowing to ReduceByKey add 48b6ce8 #! [euphoria-flink] Suppress unchecked warnings add ce0526b #! [euphoria-flink] Fix attached window emission (was too early) add 057f9fe #! [euphoria-flink] Stabilize test add 91f7c20 #! [euphoria-flink] Fix label for time-sliding windows on flink add 6dc35df #! [euphoria-flink] Add TODO add c8e714a #! [euphoria-flink] Allow tuning window emission parameters add b415474 #! [euphoria-flink] Align flink and euphoria-core time sliding labeling add e6c282c #! [euphoria-flink] Fix streaming MapElements add 47cfdd8 #! [euphoria-flink] Tune windowing parameters for test add 69f842c #! [euphoria-flink] Descrease presure for jenkins add df6b894 #! [euphoria-flink] Descrease presure for jenkins add 247a21a #! [euphoria-flink] Correct test setting add 7b2b631 #! [euphoria-flink] Correct auto-watermark-interval setting add b3760fe #! [euphoria-flink] Have StreamingWindowedElement flow through RSBK impl add e5bca6f #! [euphoria-flink] Integrate attached-windowing into RSBK impl add 8b54c16 #! [euphoria-flink] Handle negative watermarks add 3d65d2b #! Move TimeInterval out of Time add ec36f20 #! Make TimeSliding using TimeInterval as a window label add 324b75d #! [euphoria-flink] WindowedElement to StreamingWindowedElement in join-translator add c0c18e0 #! [euphoria-flink] Drop native join translator (rely on RBSK) add 868abbc #! Drop dead comments add 71aa697 #! Centralize flink version add 49a69b0 #! review add e085956 #! moved flink operator testkit to the euphoria-flink package add 8854373 #! implemented state storage for states in inmem executor add 5b2a07a #! implemented state storage in flink, not well tested add 3372b4e #! small polishing add 52eb3f0 #! code polishing after review add 085d7b5 #! State is not required to be Serializable add faa6fe5 #! [euphoria-flink] implementation of non-combining ReduceByKey for streaming API #! #! [euphoria-flink] fix failing tests #! #! [euphoria-flink] use singletonIterator add e12fd2a #! small formatting changes add 026dcb8 #! refactored statestorage to work on storage descriptors add 384a6c3 #! changes after review add 0dab05f #! [euphoria-flink] Upgrade flink to 1.1.2 add b3083df #! [euphoria-flink] Switch time and time-sliding to euphoria implementations add 5c6a1be #! [euphoria-flink] Correct forwarding operator parallelism add aa3c515 #! [euphoria-testapps] Emit only top-200 words from word-count add e3d7421 #! [euphoria-flink] Fix memory leak on window triggers add f72188b #! [euphoria-flink] Attempt to fix clear-window-state leak in backend database add f048fb6 #! [euphoria-core, euphoria-flink] Align trigger behavior in inmem and flink add 61b4a67 #! [euphoria-flink] Testify early triggering add 04d3b70 #! [euphoria-flink] Schedule window only once add a43c3c0 #! [euphoria-flink] Rename FlinkTrigger to FlinkWindowTrigger add 493f4d7 #! [euphoria-flink] Fully integrate generic windowing add fe4db0a #! [euphoria-flink] Fix type annotation add 200120f #! [euphoria-flink] Fix after rebase add 0b904d0 #! [euphoria-flink] Reduce noise on unchecked warnings add f105d3f #! [euphoria-flink] Small refactoring add c88a579 #! Stabilize flipping test add 7540f3b #! distinct operator test from testkit moved on event time add ec61b70 #! countByKey tested by operator test in flink add 673f1c5 #! sumbykey and reducebykey on event time windows add 946b296 #! tuned operator testkit for flink, passes on batch inputs add eb6581b #! flink keys have to be comparable or exception is thrown add 307c931 #! TimeInterval is Comparable add a98b9aa #! functional all operator tests on flink add 6b82d80 #! changes after review add f829502 #! WIP: #16426 Remove GROUP from Windowing add 6a7ca2c #! [euphoria-core] Avoid needless allocation during WindowID#hashCode computation add ccac09d #! [euphoria-core] Make inmem RSBK handle groupless windows add 802d219 #! [euphoria-operator-testkit] Remove GROUP from windowing add e581a0c #! [euphoria-core] Drop global count window test (feature discontinued) add b63233c #! [euphoria-flink] Remove GROUP from windowing add 59e36b2 #! [euphoria-flink] code style add 36f270b #! [euphoria-core] Rename Collector -> Context add 8fbab4e #! [euphoria-core] Provide Context#getWindow add ee48e47 #! [euphoria-flink] Provide Context#getWindow add 6975d48 #! [euphoria-*] Drop WindowedPair add 4c16656 #! [euphoria-*] Avoid using the term "windowed pair" add daa8451 #! [euphoria-flink] Fix windowing for batched based R(S)BK translators add 11b1af2 #! forward stamp with each element in inmem executor add 266c188 #! [eurphoria-core] Added tests for earlytriggered session windowing, fixed functionality add c068600 #! [euphoria-core] FIXME: overridden time windowing for element-time add fd0b9b2 #! Passing element time within pipelines in inmem executor add 8ddff79 #! [euphoria-ios] added IOS application module add cf36cf1 #! [euphoria-ios] refactored module structure, separated server from flows add 27583e9 #! [euphoria-ios] added core for demonstration of lambda architecture add 9328678 #! [euphoria-ios review] removed several FIXMEs added test for TimeSliding windowing, correct passing of watermarks between operators add 8e3b6a9 #! Trigger API redesigned add 9c50460 #! [euphoria-inmem] ReduceStateByKey against new trigger api (merging-windows not yet supported). add 5d6d6d1 #! [euphoria-inmem] Implement MergingWindowing support add 40af773 #! [euphoria-inmem] Enable sort-state test add ec1e0fc #! [euphoria-inmem] Attempt to fix fliping test add 3329936 #! [euphoria-inmem] Make Windowing#trigger not return null add 7a77aa3 #! [euphoria-flink] Executor uses new Trigger API add e85c896 #! [euphoria-flink] Minor cleanup add d32b85d #! [euphoria-ios] Get compilation right add 8e9d794 #! [euphoria-flink] Forward element timestamp add bc2910b #! [euphoria-inmem] Fix trigger state merging add 7f1079a #! [euphoria-inmem] Create a new instance of trigger state during merge add 533db50 #! [euphoria-core] PeriodicTimeTrigger is aligned with window start timestamp add c2ebdef #! [euphoria-core] Fixed timer registration in PeriodicTimeTrigger add 77ac266 #! [euphoria-core] Drop logging in periodic-time-triggering add 4c6f1b7 #! [euphoria-inmem] Ensure calling onElement on merged windows add a1068d3 #! [euphoria-core] added serializable Supplier #! [euphoria-hadoop] added DataSourceInputFormat #! [euphoria-kafka] tuned KafkaSource to be able to read whole partition add 6413613 #! [euphoria-kafka], [euphoria-core], [euphoria-hadoop] changed life cycle of writer object, added flush method, some code polishing add cc6558c #! [euphoria-ios] review and tuning add b9d8f0a #! [euphoria-kafka] renamed all euphoria kafka artifact references to euphoria-kafka_0.10 add c77e5712 #! Upgrade to Flink 1.1.3 add ceb1387 #! [euphodia-ios] remove the module from parent pom add c6ddec4 #! [euphoria-flink] RBK: Support for merging windows (streaming) add 44b9424 #! [euphoria-flink] RSBK: Verify correct behaviour during merging-windowing (streaming) add 33bb70f #! [euphoria-core] Require MergingWindowing#W being Comparable add 70979c2 #! [euphoria-hadoop] Fix TaskID retrieval from Hadoop context add bdc8540 #! [euphoria-core] correct discarding of latecomers add 225cca9 #! [euphoria-flink] Code style; apply diamond operator add ba99334 #! [euphoria-flink] Fix non-combining RBK on bounded input add 045c9ab #! [euphoria-core] fixed latecomers - review add 3d9faac #! small fixes add 8542f16 #! [euphoria-hadoop] DataSourceInputFormat/DataSinkOutputFormat don't operate with Pair anymore add 6f25cba #! [euphoria-spark] Spark executor implementation for bounded datasets add 2701cd7 #! [euphoria-core] Add operator type annotations add e878b4f #! [euphoria-core] state complexity and number of repartitions in operator annotations add 8c55a27 #! [euphoria-ios] added module for command line tools add 806b710 #! [euphoria-ios] added description of lambda and kappa architecture, tuned applications add 2efb58d #! [euphoria-ios] added support for spark, finished presentation add 5cd5a09 #! [euphoria-ios] performance tuning of Trends application add ddc2819 #! [euphoria-ios] slightly optimized spark RSBK, IOS presentation add e4eaf9b #! [euphoria-ios] rebased on master add bcdbdc4 #! [euphoria-ios] review and removed duplicate code add d97d2cb #! [euphoria-flink] Support window merging in batch mode add 595dffe #! [euphoria-core/flink] Align element timestamp with window emission add 1d06100 #! flink batch repartition + repartition batch tests add 92bcb65 #! partitioning with default partitioner add ba72d08 #! Executor API add f8c1da9 #! inmem and flink executor implementation add 9e8556a #! spark executor add 4c85276 #! cleaning add c3a7db8 #! DefaultPartitioner as its own type add 393f37c #! [euphoria-core] Required explicit windowing if not joining batches add 121636c #! [euphoria-flink] Fix logic in evaluating RBK translation add d15606b #! [euphoria-core] Make WindowingRequiredException a top-level class for re-use purposes. add ee102db #! [euphoria-core] Javadoc; Remove reference to seznam internal ticketing system add 3c81993 #! [euphoria-core] Javadoc; Make promise about failing upon #preTranslateValidation add aead0f4 #! Event time extractor function at operator level add 4ccd6fa #! minor refactoring after review add e2a4809 #! [euphoria-testkit] Demo testkit junit integration add 0728f72 #! [euphoria-testkit] Processing annotation add 0a1b913 #! [euphoria-testkit] Add more tests to demo testkit add 41d3c89 #! [euphoria-flink] ModuloSplitAssigner for testkit add 4e37619 #! [euphoria-testkit] Use NG jUnit runner add 4c0ce01 #! Spark executor - full support of ReduceByKey/ReduceStateByKey including merging windows add 6e52322 #! Remove GroupBy operator add c770a54 #! Avoid flink stream hanging upon cancellation add c555da6 #! [euphoria-spark] Fix partitioner when partitionId is negative add 2b5578e #! TimeInterval windows are right-open intervals (not overlapping anymore) add 0840e74 #! [euphoria-hadoop] Remove MultiReader from HadoopDataSource add eb96644 #! [euphoria-hadoop] Remove SerializableWritable from public constructor add 5bfa655 #! [euphoria-flink] Use LocatableInputSplitAssigner for batch inputs add c580a95 #! README.md proposal add 70c0776 #! Prevent windowing with zero duration add 924971d #! Drop KafkaSource#batchSize #! #! The same effect is achievable through KafkaConsumer's paremeter 'fetch.max.wait.ms'. add c4861a8 #! Provide easy-to-use hadoop based data-source/-sinks add 122d69f #! Update README add f1e5e0e #! Fix few TODOs add 570bbec #! Add convenience hadoop data sink around text files add 4c10ffa #! Provide simple-hadoop-text-file-data-source add f0707ab #! Javadoc add b465e4b #! Drop "Data" from the name of hadoop data-sources/-sinks add 808b583 #! Imperative to functional style add dc188f0 #! Extract inmem executor into a dedicated module add 4c4822f #! Drop unneccessary dependency on commons-io add 8d424b7 #! Upgrade to guava 21.0 add f36ca4d #! [euphoria-testapps] Resolve conflict on kryo add a7ae3c0 #! [euphoria-flink] Use key type Tuple2 instead of ComparablePair (performance tuning) add 59a2c97 #! benchmarks unification add a977360 #! Add LICENSE (APLv2.0) add c58fe7e #! Ensure LICENSE/NOTICE files in produced artifacts/jars add 30cc4c5 #! Drop guava shading add 5de2604 #! Move euphoria-testapps to a dedicated, top-level repo add f90cabf #! Remove obsolete dependency definitions add 0fa55ce #! Upgrade maven jar plugin; Cleanup debian package dependencies #! #! Conflicts: #! debian/control add b39ec19 #! Polish README add 0b8b0fc #! Get rid of szn specific stuff add e1e21e7 #! Integrate licensing headers mechanism add f1ecb48 #! Apply license headers add 3bbf9d9 #! Update new SCM information; Drop obsolete SZN specifics add 6305c68 #! Support custom artifact deployments add e22ee28 #! Correct spelling add 3d1f8eb #! [euphoria-kafka] Properly quit read loop when interrupted add 22f557f #! Correct name of copyright owner add 08e0888 #! Temporarily remove documentation section add 4133f75 [WIP] Prepare maven central deploy (#3) add a3ccbd7 #! Link documentation from within README add 1eaee22 #! Add self-contained euphoria-examples module for demoing purposes add 26f3e0f #! Add link to examples add e535a86 WIP: Integration with travis-ci.org (#6) add d773807 #! [euphoria-spark] Upgrade to Spark 2.1.0 add 09919e9 #! Drop obsolete "hudson" profiles add 98bb4b3 #! Cleanup obsolete gitignore patterns add 06db55c #! Add Travis CI badge into README add 4e5ca08 #! Drop obsolete class add bb6c750 #! [euphoria-core] Cleanup inmem specific classes add 2f13c9e #! Sanitze javadocs add 0235355 #! Drop obsolete class add e0c7836 #16 Replace Dataset#getPartitioning() with Dataset#getNumPartitions() add e9e2be4 #16 Move Partitioning related classes to a dedicated package add 0102e6f #! Attempt automatic deployments add 860de6a #! Exclude continuous-deliver folder from being license validated add f0a5216 #! Set execution flag on deploy scripts add 12823a8 #! Add missing <name/> <description/> add b89e5e7 [maven-release-plugin] prepare release v0.5.0 add 3056333 [maven-release-plugin] prepare for next development iteration add 8bbc0ae #! Avoid running tests twice add 11d95d3 #! Mention maven dependencies in README.md add 5a9590d #8 Suppress safe unchecked warnings add db54414 #! Rename StorageDescriptorBase to StorageDescriptor add 815e3eb #18 Fix a few minor bugs after static code analysis add acb5f97 #18 Fail build when troubling error is found by FindBugs analysis add 0a945e7 #18 Use @Nullable annotation add 320aef8 #10 Improve documentation around URI based data-sources/-sinks add dc3a35a #10 Fix typos add 21ff8b4 #! Add contribution section to README add f68363a #! [euphoria-flink] Allow enabling object reuse add cdd00f8 #48 Move most maven plugins under the "regular-build" profile (active by default) add 0348e98 #! Document regular-build profile add 8d5290c #27 [euphoria-flink] Rewrite windowing to native implementation of StreamOperator add b6c27e9 #! [euphoria-flink] Avoid extra shuffle when windowing on streaming add 4a60228 #47 [euphoria-flink] Alternatively execute windowing/valueOf fully after shuffle add 2ff48b9 #! [euphoria-flink] Allow specifying latency tracking interval add d042d44 #! Make WindowedElement an interface add caab9fb #! Drop WindowedElementImpl from euphoria-core add c0218bd #! Make window-element-factory use-case specific add ad64695 #! [euphoria-kafka] No sync commit add 56f24ce #14 [euphoria-flink] Don't send timestamp along with each element. Use rather internal Flink record's timestamp. add bb056e1 #! Add link to gitter chat add b2a8b8b #14 review cleanup add b6822ad #14 [euphoria-flink] Allow WindowAssigner operator chaining add 95e97b8 #60 Prefer ValueStorage instead of ListStorage for combining RSBK operations add 1e8fb83 #! Register Either for serialization add 2f276a6 #! Suppress safe casts add 7b3de98 #! Add a dedicated interface for event-time extraction to reduce auto(-un)-boxing add d4fca7b #! Register Triple and clean up in TopByKey add 2ec232a #47 Windowing#assignWindowsToElement returns Iterable instead of Set add c314747 #47 [euphoria-core] TimeSliding window assigner generates custom SlidingWindowSet (smaller serialized size) add f2a4a85 #! Make storage descriptors static add 791ce31 #! [euphoria-flink] Do not cache states add 51f7e5b #! Drop Supplier from public API add 75e3b54 #67 [euphoria-flink] Avoid descriptors cache infinitely growing add 9070162 #67 [euphoria-flink] Avoid flushing empty states due to "localMode" add 49b59e3 #67 [euphoria-flink] Fix window merging add 032f862 #67 [euphoria-flink] Avoid inlining processTriggerResult add 9665f3d #! Include link to FAQ wiki page add 6627cff #! Add shaded-guava for internal use within euphoria add 11d4623 #! Add javadoc generation add 34eec7a #! Switch over to shaded guava add 5c04cc4 #51 Do _not_ store storage-provider in states by default add b333458 #51 Move StateFactory to the state package add 32fc452 #51 StateFactory interface revised add 59ca612 #51 Replace StateCombiner with StateMerger add de5672b #51 Drop obsolete TODO add aca00ec #51 [euphoria-core] Skip processing empty window merge sets add f14b1a1 #19 [euphoria-flink] Avoid extra shuffle (on the batch executor) add 361b67e # [euphoria-flink] Fix trigger state clean-up of merged windows add 6abaeb9 #! [euphoria-examples] Example of web server log aggregation add 65e01f0 #! [euphoria-examples] Fix license header add 930086e #61 [euphoria-flink] Unit test covering out-of-order stream with allowed lateness add 3eb2ff2 #79 [euphoria-core] Trigger#onMerge does not return TriggerResult add e83a0a8 #! InputSplit#toString - especially useful on spark add a363dbc #! Add convenience hadoop sink add c4a949d #! Generalize HadoopStringSink to accept any type of object add 535a277 #! [euphoria-flink] Warn about failing execution scenario add 018a710 #! Update copyright add 0811937 #85 Move findbugs plugin execution to the process-classes phase add bde04a0 #77 Import trends benchmarks add be64ccd #77 Drop obsolete shell script add b7788cd #77 - parsing fix add ddee475 #! Variable year in license header add 7a7a7e7 #! [euphoria-core] Avoid clearing count state twice add 651b8a1 #! Fix typo add f236fa0 #95 Rename Batch to GlobalWindowing add 3193136 #! Drop Map.Entry from Pair add b400b24 #! Add @Experimental for easier API evolution add 579e349 #21 [euphoria-core] Code/Javadoc style add 7aeded1 #21 [euphoria-core] Javadocs for FlatMap add 4d2f182 #21 [euphoria-core] Javadocs for Union add ceb554a #! Add contact information to README.md add d58c055 #99 Sort operator support add 6e97137 #21 [euphoria-core] Javadocs for Repartition add c1d7dd1 #21 [euphoria-core] Javadocs for ReduceStateByKey add 0afdef8 Update README.md add 7d18201 Update README.md add e98a2f5 #! Remove obsolete class add 87d2a9c #! Do not allow derivations from Pair/Triple add a23f6ae #98 [euphoria-flink] Do not allow object re-use add 247940d #! - javadoc refactoring add ca54f8b #13 [euphoria-flink] Batch executor uses hash code as a key in shuffles (IntComparator performs much better) add 46fd584 #13 [euphoria-flink] Fix bug in batch ReduceByKeyTranslator and add unit test add 90862ba #13 Fix race condition in batch ReduceStateByKeyTranslator add e43bfe6 #13 [euphoria-flink] Copy input instance in ReduceByKeyTranslator because of Flink's object reuse add 12fbd5e #! some corrections/formatting add 1c83ceb #13 [euphoria-flink] Avoid hash code collisions in batch ReduceByKeyTranslator add fa2bcfd #105 Add (failing) proof-of-concept test add 4cebcec #105 Rework StateFactory API; Stabilize JoinState implementation add 27789b8 #43 Annotate experimental Session#earlyTriggering add f5d9cb8 #105 Do not read entire disk storage into memory add bb3f3c0 #105 Provide an experimental "early emitting" join state implementation add e75738c #105 Simplify closing spilling file add b9c46a8 #105 Document some more experiment dangers add 6a6ac4b #! [benchmarks/native-flink] Filter out invalid entries add 01b64b7 #! Upgrade to flink 1.2.1 add 70f64af #! Prevent deleting wrong timers add d249f85 #! polish: removed unnecessary null check, variable reuse add f5691f4 #105 Add reference to future solution add 1fe81fe #! Prepare for 0.6.0 release add d2b45a2 [maven-release-plugin] prepare release v0.6.0 add d823321 [maven-release-plugin] prepare for next development iteration add 10a9bcb #! Prepare for next dev iteration on master add d849eaf #! Update README regarding new release add 65d9cd4 #! Fix loading configuration add d877dc9 #! [euphoria-examples] Explain in details the word-count-example add f58ee3c #! [euphoria-examples] Explain in details the access-log-count add 29849fc #! [euphoria-examples] Correct typos add c4469af #! [euphoria-flink] Stabilize test add 4c0bd91 #! Stabilize sort-test add 2b20086 #! Stabilize count-by-key-test add 064cdbe #119 [euphoria-core/-inmem] Rework event time assignment add e9b39a9 #119 [euphoria-flink] Rework event time assignment add c1ff480 #119 [euphoria-spark] Rework event time assignment add 9110af2 #119 Add missing license headers add 1fa0516 #119 [euphoria-examples] Rework event time assignment add 35a432c #119 [euphoria-benchmarks] Rework event time assignment add ddd35d4 #! Stabilize inmem-executor join test add fa371b4 #121 Allow multiple elements to be emitter from ReduceByKey add c4b1cda #121 changes requested by review add 2df1c7b #121 fixed comment add cc3737e #113 [euphoria-spark] Implement disk splitting list-storage implementation add 5923734 #113 [euphoria-flink] Unify disk spilling implementation for list-storage add 0aa8aad #113 Add missing license headers add 15d0726 #113 [euphoria-spark] Drop obsolete warning suppression add db43b35 #113 [euphoria-testkit] Enable accidently disabled RBK tests add a1c0d16 #113 [euphoria-core] Support try-with-resources for Serializer.Output/InputStream add e50386e #113 [euphoria-testkit] Allow different local spill file allocation strategies add efe71ae #113 [euphoria-flink] Support a wider area of types during (de-)serialization in list-storage add 17a9e13 #113 [euphoria-core] Testify FsSpillingListStorage add f680510 #113 Add missing license header add d8c5ffa #31 [euphoria-core] Implementation of accumulator API + integration with Flink executor add 62696db #31 Refactored according to review feedback add f851540 #133 [euphoria-spark] Accumulator API supported in Spark executor add 7b65c77 #133 [euphoria-inmem] Accumulator API supported in in-mem executor add 5518ca3 #133 [euphoria-spark] Collector should be transient in UnaryFunctorWrapper add a8ce723 #135 Add SingleJvmAccumulatorFactory add 57a232c #135 Demostrates usage of accumulators validation in operator testkit add f8f26b4 #135 Fix histogram accumulation add 56a041e #135 GetSnapshot -> Snapshotable add 9ce7fd6 #135 Add tests covering accumulator usage add d522ffd #135 Fix failing tests on the spark executor add d5866bf #134 [euphoria-spark] Implemented support for Spark native accumulators add f6b671f #134 [euphoria-core] Fix misleading log warning in VoidAccumulatorProvider add 32ac9ff #134 [euphoria-spark] Clone accumulator factory before flow execution add 4318a23 #134 [euphoria-spark] Make SparkAccumulatorHolder thread-safe add 135c828 [maven-release-plugin] prepare release v0.7.0 add ebf571f [maven-release-plugin] prepare for next development iteration add 0624b1d #! Version up add e24d83c #! Fix Javadoc warnings add 8fbe530 #! Fix Findbugs warnings add 43b6548 #! [euphoria-spark] Correctly transfer global state to executed UDFs add 52bd0c8 #! Pete dropping out add db08931 [maven-release-plugin] prepare release v0.7.1 add b5479cb [maven-release-plugin] prepare for next development iteration add 975c82c [euphoria-spark] Propagate exception in SparkFlowTranslator add fedb7e6 [euphoria-hadoop] fix SequenceFileSource add 667c0bb [euphoria-hadoop] wrap hadoop config in SequenceFileSource, so it cannot be changed by user later on. add 5bbfe22 [euphoria-core] Remove IORegistry. add d7ce113 [euphoria] add links to license-header.txt from subprojects add fbccc54 [euphoria] exclude license-header.txt itself from the license header check add 5ac317c [euphoria-flink] #159 fix default parallelism in local flink runner to 4 add d97b4fd add support for Join operator hints add 39b8e20 [euphoria-core] #160 remove partitioning from stateful operators, bump version to 0.8-SNAPSHOT add 4a8582f [euphoria-core] #160 drop explicit partitions from ListDataSink add 40f5631 [euphoria-operator-testkit] #160 compilable testkit after partitioning removal add fbc8d07 [euphoria-kafka] #160 module compilable after changes add 40a124b [euphoria-hadoop] #160 module compilable after changes add 43d8e60 [euphoria-examples] #160 module compilable after changes add 1933624 [euphoria-flink] #160 module compilable after changes add 657ffe3 [euphoria-fluent] #160 module compilable after change add 4b31726 [euphoria-core] #160 removed partitioning and repartition operator add 732e2c3 [euphoria-inmem] #160 compilable, but with failing tests add a31d433 [euphoria-inmem] #160 more tests working after code change add 9a17043 [euphoria-inmem] #160 build passing tests after changes add 08193e9 [euphoria-spark], [euphoria-flink] #160 modules compilable after partitioning removal, still some tests failing add 0729758 [euphoria] #160 build passing including tests add da27b28 [euphoria] #160 remove mentions of partitions where no longer makes sense add 1eb4891 [euphoria-hadoop] remove sneakythrows add 1ee8f27 [euphoria-kafka] remove groupid from kafkasource add d12fa50 [euphoria-benchmarks] make benchmarks part of the build add 9270f2a [euphoria-operator-testkit] activate mistakenly ignored test add 30b567c [euphoria-core] add Audience annotation add fb40e00 [euphoria-core] remove checkpointing from datasets - unusable feature add 75aeae7 added IOUtils to help iterate over methods throwing IOException add 3384c56 added license add f1fe96f [euphoria-core] #158 add `withSortedValues` to `ReduceByKey` operator add 9c22026 [euphoria-core] #158 basic non configurable implementation of value sorting add 32048a0 [euphoria-core] #158 add StateContext and validate on inmem executor add e841fa6 [euphoria] #158 implemented external sorter for flink and spark add 07ec729 [euphoria-core] #157 implement possibility for DataSink to modify output flow before persisting it add be78df9 [euphoria-core] #157 review add b0811cf [euphoria-local] #173 changed InMemExecutor to LocalExecutor add 3d35443 [euphoria-core] #177 added MultiDataSink add 1582804 [euphoria-core] #177 MultiDataSink renamed vars add e36e8e8 Move to gradle. add e859bff Add euphoria-examples module to gradle add 6f72497 Do not ignore spotbugs failures. add 9aaef68 [euphoria] add maxTimestamp to generic Window add 83b6caa [euphoria-local] fix findbug warning regarding random reuse add dfdbf28 [euphoria] #156 distinguish batch and stream sources add 4536804 [euphoria-core] #156 Add split to boundedsource add 44fe36a [euphoria-core] #161 replace Iterable with Stream in reducing operations add 816b570 [euphoria-local] fix non deterministic test add a3ced9b Fix javadoc. add f7d7148 [euphoria-core] DataSinkWrapper is now Serializable add 40c56e4 Shade guava add 7e51c17 Fix shadow jar publishing. add d06b7af Fix guava add f14b660 [euphoria-build] remove unused protobuf plugin add 62f6ac9 [euphoria-core] bugfix building of RBK with no windowing add d9b4d9d [euphoria-hbase] #181 add HBase module add 27669fa #181 Code review. add 16435f7 [euphoria] add compileOnly to testCompile add 0bbfcc3 [euphoria-core] Union can accept more than two data sets add 74ef093 Update readme. add 2f65b8f [euphoria-operator-testkit] Reduce by key bounded parallel test. add 23233b6 [euphoria-operator-testkit] Code review. add 30b216d [euphoria-core] Join operator API improvements. add 723ce33 [euphoria-core] Join operator hint tests. add 9eed8d6 [euphoria-core] Join operator hint tests - equals, hash code. add 59c9613 [euphoria-core] Join operator. Cleanup. add cdf583b [euphoria-core] Code review. add f0a4a0f [euphoria-hbase] #181 add bulk loading to hbase sink add 8721b96 [euphoria-spark] broadcast join. add c728ebb [euphoria-spark] broadcast join code review. add 3a42689 [euphoria-core] Publish test jar. add 8c636a2 [euphoria-hadoop] wrapping hadoopConfig in SequenceFileSink add 27cf3d3 Added outputValues() to operators producing pairs add 98c2820 Updated tests for outputValues add 591b035 [euphoria-core] Test ouputValues on all executors. add 91eacce [euphoria-core] make sure that windowBy() can be appliedIf() add 6301917 [euphoria-hbase] HFileSink uses raw types add b957f4b Fix javadoc. add bbfc3e7 [euphoria-hbase] add windowing to hfilesink add f8b6683 [euphoria-flink] move to flink 1.4 add 3e6e1ea [euphoria-core] Optimize ReduceWindow operator for attached windowing add 2ff9d12 [euphoria-hadoop] add failing test case add 0a36377 [euphoria-hadoop] Refactoring, tests and fixes. add 84d43ae [euphoria-hadoop] Check output specs during HadoopSink initialization. add cdea5a1 [euphoria-hadoop] Update exception and hadoop utils. add ddb3ce6 [euphoria-sink] Fix test. add f54c333 [euphoria-hadoop] Test DataSinkOutputFormat. add 2e0fc3b [euphoria-hadoop] Make sure initialize is called only once. add 115fd48 [euphoria-hadoop] Make sure initialize is called exactly once. add 756999e [euphoria-hadoop] Code review. add 85b0e83 [euphoria-hadoop] Code review. add 3b3fc4f Bump gradle version add 93c2501 [euphoria-core] remove getDefaultParallelism from BoundedDataSource add 1a25709 [euphoria-testing] Init. add 97040e1 [euphoria-testing] Add comments. add 15ce525 [euphoria-testing] Code review. add ec149fb [euphoria-core] performance tuning add 309fef7 [euphoria-core] add test for applyIfNot add e170004 [euphoria-hadoop] small fixes add be66084 [euphoria-hbase] fix bulkloading add a4def15 [euphoria-spark] register default classes add 05df1df [euphoria-spark] Introduce executor builder. add 2ddf200 [euphoria-spark] Simplify. add 9db0c6b [euphoria-flink] register common classes by default add 89b3ab0 [euphoria-spark] fix build add 9892a06 Update README.md add e3750c8 [euphoria-spark] Add hash map to default classes for kryo. add 6a18f4a [euphoria-spark] Add default parallelism to local mode. add 776d856 [euphoria-core] #21 add builder javadocs to operators add fc79a6b [euphoria-core] Fix javadoc. add bbdc9bf [euphoria-hadoop] InputSplit doesn't have to be Writable add ace7310 [euphoria-examples] add hfileloader add 965b5d5 [euphoria-examples] Code review add 176303c Happy new year!! add 5de2def [euphoria-hadoop] SequenceFileSink builder add 60d3572 [euphoria-hadoop] SequenceFileSink builder refactor add b757249 [euphoria-hadoop] sequencefilesink builder consistent with other builders" add 352aa55 added api example add 9a0097c [thirdparty-guava] use shaded jar with shadow classifier add 68ad116 [euphoria] fix javadoc and unchecked warnings add 98220cd [euphoria-build] add maven build for releasing add 1d68700 [euphoria-hadoop] fix build add 78f3e62 [euphoria-hbase] make sure hfiles have correct permissions before bulkloading. add 1b4d6d8 Hbase bulk load add ALL permissions for Hbase user add f577b94 [euphoria-flink] flink register classes with serializers add 157830b check kafka partitions availability add 36b8a3b Check if leader is available in KafkaSource add 6fdfb48 Correct indentation add c8612fa Test KafkaSource reaction to no consumer partitions add b25a0e2 add licence to KafkaSourceTest add 695bb23 Add tests in KafkaSourceTest add b983cad Add tests in KafkaSourceTest 2 add 7c88a96 Fix KafkaSourceTest add c2b9941 [euphoria-flink] #260 Flink - broadcast hash join add 368b903 [euphoria-flink] #260 reformatting and corrections add ae149c4 [euphoria-flink] #260 small changes add 63eae04 Upgrade kafka to 1.0.0 add 0f8c060 javadoc fixes add c761e52 [euphoria-spark] #243 RDD are cached when used more than once in flow add 42194a4 [euphoria-core] #259 Hints are not runtime specific. Hints describe OutputDataset add e5ca99e [euphoria-core] #259 Hints are now in operator, not Dataset add 8cfc753 broadcastHashJoin fix add f707c2d broadcastHashJoin fix - util class renamed add a1ffe21 AVRO read example add 965f50d sq add 55f5568 Fix code in README.md add 96b0efd [euphoria-examples] better exception handling in SimpleWordCount and README.md add 32b3d73 [euphoria-spark] Optimize kryo serialization. add c5fffe8 [euphoria-spark] Singleton serializer test. add 2f0474b [euphoria-spark] Synchronize singleton serializer. add 2c5a872 Add support for expensive computation hint. add 14e959c [euphoria-spark] Fix RDDCaching test. add 9abe142 [euphoria-spark] Improved RBK translation. add 14b345c [euphoria-spark] Fix cache RDD test. add c4e71b4 [euphoria-spark] Remove timestamped element. add abcf798 Core changes. add b0b79d6 beam wip add 4f921cd Rebase on master add f2a9a6c wip add 57e500b wip add a674bf1 wip add 193e61f WIP: RBK non deterministic, need tuning of FlowUnfolder add b48e4f3 [euphoria-beam] PoC of beam integration add 1f7cd50 [euphoria-beam] add Dataset#persist() listener add 43c79ee [euphoria-beam] cleanup add 2fe1db2 [euphoria-beam] correctly pass coders from inputs add 3c422c7 sq add 4f876d6 [euphoria-beam] BeamFlow is forced to be backed up by Pipeline add 96fbd8c [euphoria-beam] add trigger to window in RBK (missing correct trigger wrapper) add 6ece500 [euphoria-beam] bump beam to 2.4.0 add 2d7290c [euphoria-beam] annotations are provided add 8bf22ab Missing test dependencies added. Beam version updated to last release '2.4.0'. add 5366452 Small change to fix error InteliJ Idea is wrongly reporting. add 578b133 Wrong merge during rebasing fix. add 5b72b01 [euphoria-beam] wip add 53a6fc0 First attempts to run ReduceByKey tests in BeamOperatorsTestSuite. Some of them are not running due to missing triggers translations. Others use unsupported ReduceStateByKey. add d50bad0 Small typo fix. ReduceByKeyTest in BeamOperatorsSuite disabled again. add 851bdf7 [euphoria-beam] beamphoria corrections add c3cf94e [euphoria-beam] Avoid unneccessary lamda allocation with each element in DoFnCollector. add c0c20d4 [euphoria-beam] Add tranformation names. add dcb413d [euphoria-spark] Fix windowing signature in RBK translator. add e464965 beamphoria integration part 1 add 53f9594 [BEAM-3900] removed old licenses add 7c3f686 [BEAM-3900] integrating to gradle build system add 458c625 [BEAM-3901] code reformat to comply beam code standard add 337925b [BEAM-3901] findbugs fixes add cd31061 [BEAM-3902] euphoria-core checkstyle corrections add 43194d3 BEAM-3900 Code in euphoria-local package modified to comply with checkstyle. add a3a47b9 BEAM-3900 Code in euphoria-testing package modified to comply with checkstyle. add e7342c9 [BEAM-3902] euphoria-core checkstyle corrections 2 add 7bdb20b [BEAM-3902] euphoria-fluent checkstyle corrections add d30d049 [BEAM-3900] Code in euphoria-operator-testkit package modified to comply with checkstyle. add 65d0f0a [BEAM-3900] Code in euphoria-operator-testkit package modified to comply with checkstyle. add 99314b2 [BEAM-3900] Code in euphoria-beam package modified to comply with checkstyle. add a529708 [BEAM-3902] Fix license headers and remove unused poms. add 897e93b [BEAM-3902] Move euphoria to apache namespace. add 7f1f5aa [BEAM-3902] Move euphoria to apache namespace. add abf56a2 [BEAM-3900] Allow for Beam windowing to be set through Euphoria API. add 5702cb5 [BEAM-3900] Missing licence header added. add d7cc7c7 [BEAM-4294] Join translator empty shell added. add 844c245 [BEAM-4294] Crude Join operator translation implemented. Still work in progress. add 8c00c26 [BEAM-4294] Inner join translation implemented. add dbed9fe [BEAM-4294] Left, right and full joins implemeted. add 13d0131 [BEAM-4294] BeamOperatorsSuite now includes tests of Join operator. Application of windowing in JoinTranslator fixed. add 605827c [BEAM-4360] ReduceStateByKey tests adaptation to beam. - Work in progress. add affd193 [BEAM-4360] Most tests are passing. Some are still ignored mostly due to yet unsupported features. add 6bc4227 [BEAM-4294] [BEAM-4360] Join translation and ReduceByKey test suite where moved to org.apache.beam.* package. Small imports fixes after rebase to seznam/beam eff3ffd5d224c6628606b355e3a5b6d8bc40150b. add 234e131 [BEAM-4294] [BEAM-4360] Checkstyle warning fixes. add 666d791 [BEAM-4394] Pull request review fixes. add 0d4174b [BEAM-4294] [BEAM-4360] Pull request review fixes. OperatorTranslator extended to allow more than one translator per operator. add 620c6c5 [BEAM-4294] [BEAM-4360] Circular dependency fix. add 021d582 [BEAM-4294] [BEAM-4360] Missing licenses added. add 37ebc71 [BEAM-4294] [BEAM-4360] Fix of local executor module compile error. add 9768211 [BEAM-4410] added BroadcastJoinTranslator add daa2cb7 [BEAM-4410] checkstyle fix add 2322fd1 [BEAM-4419] Update Operator Testkit Testkit was moved into 'euphoria-beam' package. Testing instrumentation was simplified. add 81d4b59 [BEAM-4366] Two sources of test flipps found and fixed. First was wrong call to 'writer.commit();' in BeamWriteSink's method marked as '@Teardown'. It was moved to method marked '@FinishBundle'. Writters are now created per bundle. Second cause of test flips was NPE when acessing carbage-collected key from WeakHashMap in ListDataSource. The ListDataSource was redesigned so the WeakHashMap is not used at all. add 18df925 [BEAM-4366] Dynamic dependency versions. add 7ad7da2 [BEAM-4366] Fully serializable `Pair` (including key and value) introduced. No braking API changes where necessarry. Final `Pair` class was made abstract with two subclasses `SimplePair` and `SerializablePair`. add 46ec454 [BEAM-4443] Builders of Join operators extended to be able to expose windowing settings. Project do not compile. Still work in progress. add 571fa65 [BEAM-4443] Builders of ReduceByKey operators extended to be able to expose windowing settings. Work in progress still. add eeb34f0 [BEAM-4443] Builders of ReduceStateByKey operators extended to be able to expose windowing settings. add 53e84f4 [BEAM-4443] Windowing builders now implements Builders.* interfaces which enshures proper chaining. add c0f5166 [BEAM-4443] Builders of CountByKey and SumByKey operators extended to be able to expose windowing settings. add 8dd2822 [BEAM-4443] Windowing in parameters of operator builders unified. add a062b3a [BEAM-4443] Builders of Distinct operator extended to be able to expose windowing settings. add 9e46e59 [BEAM-4443] Builders of ReduceWindow operator extended to be able to expose windowing settings. add 18189b5 [BEAM-4443] Builders of TopPerKey operator extended to be able to expose windowing settings. add 831dd6e [BEAM-4443] Euphoria to Beam translators updated to reflect changes in windowing. add 7e8f994 [BEAM-4443] Tests adapted to new windowing API. add 76af171 [BEAM-4443] Euphoria's local executor removed. add f5054d3 [BEAM-4443] Build fixes (checkstyle, failing tests, findbugs). add be5eeda [BEAM-4443] All `WindowBy` builders now contains backward compatible `windowBy` method to define Euphoria winowing. add d97d89e [BEAM-4443] All `windowBy` builders now implements `OptionalMethodBuilder`. add f953832 [BEAM-4443] Javadods updates. Several classes moved to more fitting packages. add fd08754 [BEAM-4478] Code review fixes. add 7567121 [BEAM-4478] Accumulators translation to Beam Metrics add 1645ce1 [BEAM-4478] added operator name as namespace add b7b8955 [BEAM-4478] corrections add e3f2c19 [BEAM-4478] Second review fixes. add 65347a6 [BEAM-4529] MapElementsTest, DistinctTest, CountByKeyTest, SumByKeyTest, FilterTest enabled. Small modifications where necessary to make them work. TopPerKeyTest could not be enabled since TopPerKey is translated into unsupported ReduceStateByKey operator. WindowingTest need to be redesigned. WatermarkTest was removed. add ed392f1 [BEAM-4560] `PTransform` can now be described in Euphoria API (`BeamPTransform`). add c207983 [BEAM-4529] Typo fix. add 69ec073 [BEAM-4530] Executors concept removed. add db74781 [BEAM-4530] The `BeamMetricsTranslationTest` modified to run without executor after rebase above 'dsl-euphoria'. add 63e5b9f [BEAM-4530] Code review fixes. add 35d13a8 [BEAM-4428] added Javadoc and updated README add 127fb65 [BEAM-4428] `BeamFlow.create()` renamed to `BeamFlow.of()`. `BeamExecutorContext` renamed to `TranslationContext`. add fb95b4a [BEAM-4428] `euphoria-beam` -> `euphoria-core` package merge. All clasess from `euphoria-beam` moved to `...euphoria.core.translate` package. add 1f80c76 [BEAM-4429] add type hint add 8a31779 [BEAM-4429] added typeHints to valueBy add 50a434b [BEAM-4429] TypeHint changed to TypeDescriptor add 0212099 [BEAM-4609] Race condition was removed from translation of combinable `ReduceByKey` operator. add 357c14b [BEAM-3900] Circular dependency fix. add ac00891 [BEAM-3900] Adaptation of build process after rebase above beam/master. All `build.gradle` files in euphoria modules adapted. Some compilation warnings removed, `@SuppressFBWarnings` removed due to licensing issues, unused utils class `OperatorTranslator` removed. add 45e23bf [BEAM-3900] Formating changes done by `./gradlew spotlessJavaApply`. Euphoria-core build adapted after rebase on beam/master. add 7564a98 [BEAM-3900] Compilation warnings resolved. Some remnants of euphoria windowing removed. Minor code improvements. add e96324b [BEAM-3900] `testArtifact` configuration swapped for `shadowTest` as suggested in code review. add f05be4b [BEAM-3900] Fix javadocs. add f5755eb [BEAM-4478] Accumulators translation to Beam Metrics add 59401ae [BEAM-4585] New `RegisterCoders` class introduced to allow convinient `Coders` registration. Default `ClassAwareKryoCoder` added. add 58d8a5b [BEAM-4585] `BeamFlow` now exposes new methods to register `Coders`. Missing license headers added. add dbeacd3 classAwareKryoCoder failing test add 40f9b40 [BEAM-4705] `ClassAwareKryoCoder` fixed to not colide with other coders. add ae167f7 [BEAM-4585] Some changes required by code review. Other items will be adressed in BEAM-4705. add b6b2519 [BEAM-4585] Spotless autoformating. add ef75cec [BEAM-3900] Spotless autoformating. add fc60b78 [BEAM-4705] Kryo integration was enhanced. Api needs review since it is brand new. add 4bb0a46 [BEAM-4705] Failing test added to prove bug in functionality of `KryoFactory`. add a2f65a7 [BEAM-4705] `KryoRegistrar` wrapped into `IdentifiedRegistrar` in order to retain identity information after (de)serialization. add 5c7de05 [BEAM-4705] Spotless auto formatting. add daae838 [BEAM-4851] All operators now carry their output and key (if applicable) type. Type aware wrappers where removed where possible. add 9c67828 [BEAM-4851] All operators now carry their value type (if applicable). add 6ad8759 [BEAM-4851] So far unsucessfull attempt to use return type extracted from lambda instance. add 273f233 [BEAM-4851] Test to assert type parameters propagation added. add c429b09 [BEAM-4851] Spotless auto formatting. Compilaion warnings fixed. add 12dbc7b [BEAM-5077] Translators now use type information from `Operators` whenever possible. Way coders are resolved was refactored. add 61941d8 [BEAM-5099] `Pair` and `PairCoder` removed in favor of `KV`. Unnecesarry mapping from `Pair` to `KV` in `ReduceByKeyTranslator`removed. Some other references to `Pair` in comments and methods or variable names modified to `KV`. add 01dd754 [BEAM-5123] Wrong implicit classes imports from the same package in `OperatorTestSuite` fixed by moving it. It forces explicit imports. Some new info level logs added. add 0653428 [BEAM-5124] Documentation examples added. Join windowing check removed since it is both not working with `BeamFlow` and unnecessary due to translation to Beam. Small documentation and test changes. add 2729865 [BEAM-5124] Minor redeability improvement. add 16b4d78 [BEAM-5160] Failing test fixed. Another test to confirm the fix added. add 9e154e8 [BEAM-5161] FindBugs enabled. All warnings sorted out. Some remnants such as `ClassAwareKryoCoder` removed. add dde6a43 [BEAM-5124] Minor redeability improvement. add 1a7ca8e [BEAM-3900] Cleanup, spotless formating and test adaptation. add b2e39cb [BEAM-5195] Decomposition of `TopPerKey` was fixed. Documentatioin example and test added. add d266840 [BEAM-5205] `KryoCoder` now use Kryo's chunked input and output. add 86ce161 [BEAM-5205] Improved logging to make it easier to debug problems with Kryo registrations. add d52d343 [BEAM-5220] TODOs resolved whenever possible withing several minutes time span. Unused classes removed. add e9024f3 [BEAM-5245] Error message improved. New tests added. add 9452c72 [BEAM-5248] Translators modified so they do not set coders to input `Pcollections`, but do set coders to the outpus `Pcollections`. add 178736b [BEAM-5276] `ListDataSource` now supports data supplier to lazy fetch data when needed. That avoids serialization of data items, only the supplier is serializable. add 51ad3e7 [BEAM-5287] Attempt to build `ReduceByKey` operator with sorted values ends with `UnsupportedOperationException`. add 8ef007e [BEAM-5263] `SingleValueCollector` now supports accumulators. add eaf2fbc [BEAM-3900] Javadoc, typos, formatting, missing file fixes. add 0edb57a [BEAM-5306] User defined join functions now can ouputs more than one element. add 5df6feb [BEAM-3900] Cleanup before merge. add 02da6c1 [BEAM-3900] Auto formating, CheckStyle and FindBugs fixes. add f5f7f3c Merge pull request #46 from seznam/vasek/build-fix add 3e7f61d [BEAM-3900] Build and test fixes. add ff23186 Merge pull request #48 from seznam/vasek/kryo-build add 981369b [BEAM-5124] Documentation examples update. add 0c1e681 Merge pull request #47 from seznam/vasek/docs-examples-new add 09b621b [BEAM-5124] Euphoria API documentation updated. add a1df1d2 Merge branch 'vasek/euphoria-docs' into dsl-euphoria add edbc2e2 [BEAM-5124] Readme update. add d5d0778 [BEAM-3900] CountByKey TypeDescriptor fix add c58e3df Merge pull request #45 from seznam/simunek/typeDescriptorFix add 526a6d8 [BEAM-3900] Compilation warning fixes. add ad6875f [BEAM-3900] Unwanted edits removed. add 7071fa4 Merge pull request #6601: [BEAM-3900] Euphoria API No new revisions were added by this update. Summary of changes: sdks/java/extensions/euphoria/README.md | 36 + sdks/java/extensions/euphoria/build.gradle | 48 ++ .../core/annotation/audience/Audience.java | 45 ++ .../core/annotation/audience/package-info.java | 20 + .../euphoria/core/annotation/operator/Basic.java | 37 + .../euphoria/core/annotation/operator/Derived.java | 37 + .../core/annotation/operator/Recommended.java | 43 ++ .../core/annotation/operator/StateComplexity.java | 40 + .../core/annotation/operator/package-info.java | 22 + .../core/annotation/stability/Experimental.java | 44 ++ .../core/annotation/stability/package-info.java | 19 + .../core/client/accumulators/Accumulator.java | 29 + .../client/accumulators/AccumulatorProvider.java | 87 +++ .../euphoria/core/client/accumulators/Counter.java | 35 + .../core/client/accumulators/Histogram.java | 40 + .../euphoria/core/client/accumulators/Timer.java | 47 ++ .../accumulators/VoidAccumulatorProvider.java | 142 ++++ .../core/client/accumulators/package-info.java | 19 + .../euphoria/core/client/dataset/Dataset.java | 109 +++ .../euphoria/core/client/dataset/package-info.java | 22 + .../core/client/functional/BinaryFunction.java | 42 + .../core/client/functional/BinaryFunctor.java | 30 + .../functional/CombinableBinaryFunction.java | 25 + .../functional/CombinableReduceFunction.java | 28 + .../client/functional/CompositeUnaryFunction.java | 44 ++ .../euphoria/core/client/functional/Consumer.java | 34 + .../core/client/functional/ExtractEventTime.java | 35 + .../core/client/functional/ReduceFunction.java | 28 + .../core/client/functional/ReduceFunctor.java | 64 ++ .../client/functional/SameArgsBinaryFunction.java | 26 + .../euphoria/core/client/functional/Supplier.java | 39 + .../core/client/functional/TernaryFunction.java | 29 + .../core/client/functional/UnaryFunction.java | 50 ++ .../core/client/functional/UnaryFunctionEnv.java | 42 + .../core/client/functional/UnaryFunctor.java | 39 + .../core/client/functional/UnaryPredicate.java | 25 + .../core/client/functional/VoidFunction.java | 29 + .../core/client/functional/package-info.java | 19 + .../euphoria/core/client/io/Collector.java | 44 ++ .../euphoria/core/client/io/Context.java | 24 + .../euphoria/core/client/io/Environment.java | 54 ++ .../euphoria/core/client/io/package-info.java | 19 + .../euphoria/core/client/lib/Euphoria.java | 51 ++ .../extensions/euphoria/core/client/lib/Split.java | 120 +++ .../euphoria/core/client/lib/package-info.java | 19 + .../core/client/operator/AssignEventTime.java | 165 ++++ .../core/client/operator/CompositeOperator.java | 39 + .../euphoria/core/client/operator/CountByKey.java | 287 +++++++ .../euphoria/core/client/operator/Distinct.java | 299 ++++++++ .../euphoria/core/client/operator/Filter.java | 161 ++++ .../euphoria/core/client/operator/FlatMap.java | 253 ++++++ .../euphoria/core/client/operator/FullJoin.java | 167 ++++ .../euphoria/core/client/operator/Join.java | 345 +++++++++ .../euphoria/core/client/operator/LeftJoin.java | 166 ++++ .../euphoria/core/client/operator/MapElements.java | 190 +++++ .../euphoria/core/client/operator/ReduceByKey.java | 507 ++++++++++++ .../core/client/operator/ReduceWindow.java | 457 +++++++++++ .../euphoria/core/client/operator/RightJoin.java | 167 ++++ .../euphoria/core/client/operator/SumByKey.java | 326 ++++++++ .../euphoria/core/client/operator/TopPerKey.java | 413 ++++++++++ .../euphoria/core/client/operator/Union.java | 167 ++++ .../core/client/operator/WindowBuilder.java | 121 +++ .../core/client/operator/base/Builders.java | 207 +++++ .../core/client/operator/base/Operator.java | 55 ++ .../operator/base/OptionalMethodBuilder.java | 60 ++ .../core/client/operator/base/ShuffleOperator.java | 62 ++ .../core/client/operator/base/WindowAware.java | 38 + .../core/client/operator/base/package-info.java | 19 + .../core/client/operator/hint/ComputationHint.java | 28 + .../core/client/operator/hint/OutputHint.java | 25 + .../core/client/operator/hint/SizeHint.java | 30 + .../core/client/operator/hint/package-info.java | 19 + .../core/client/operator/package-info.java | 19 + .../euphoria/core/client/type/TypeAware.java | 52 ++ .../euphoria/core/client/type/TypeAwareness.java | 32 + .../euphoria/core/client/type/TypeUtils.java | 123 +++ .../euphoria/core/client/type/package-info.java | 19 + .../extensions/euphoria/core/client/util/Fold.java | 81 ++ .../extensions/euphoria/core/client/util/Max.java | 47 ++ .../extensions/euphoria/core/client/util/Sums.java | 42 + .../euphoria/core/client/util/Triple.java | 72 ++ .../euphoria/core/client/util/package-info.java | 19 + .../core/translate/AbstractJoinTranslator.java | 71 ++ .../core/translate/BeamAccumulatorProvider.java | 181 +++++ .../translate/BroadcastHashJoinTranslator.java | 146 ++++ .../euphoria/core/translate/EuphoriaOptions.java | 71 ++ .../euphoria/core/translate/ExtractKey.java | 68 ++ .../euphoria/core/translate/FlatMapTranslator.java | 101 +++ .../euphoria/core/translate/JoinTranslator.java | 260 +++++++ .../core/translate/LazyAccumulatorProvider.java | 70 ++ .../euphoria/core/translate/OperatorTransform.java | 87 +++ .../core/translate/OperatorTranslator.java | 55 ++ .../core/translate/OperatorTranslators.java | 33 + .../core/translate/ReduceByKeyTranslator.java | 195 +++++ .../core/translate/SimpleTranslatorProvider.java | 81 ++ .../core/translate/SingleValueContext.java | 105 +++ .../core/translate/TranslatorProvider.java | 43 ++ .../euphoria/core/translate/UnionTranslator.java | 37 + .../translate/collector/AdaptableCollector.java | 88 +++ .../core/translate/collector/CollectorAdapter.java | 40 + .../translate/collector/SingleValueCollector.java | 74 ++ .../core/translate/collector/package-info.java | 20 + .../euphoria/core/translate/package-info.java | 20 + .../sdk/extensions/euphoria/core/util/IOUtils.java | 62 ++ .../euphoria/core/util/InstanceUtils.java | 57 ++ .../euphoria/core/util/package-info.java | 19 + .../euphoria/core/client/lib/SplitTest.java | 84 ++ .../core/client/operator/CountByKeyTest.java | 130 ++++ .../core/client/operator/DistinctTest.java | 114 +++ .../euphoria/core/client/operator/FilterTest.java | 52 ++ .../euphoria/core/client/operator/FlatMapTest.java | 95 +++ .../euphoria/core/client/operator/JoinTest.java | 338 ++++++++ .../core/client/operator/MapElementsTest.java | 87 +++ .../core/client/operator/OperatorTestUtils.java | 76 ++ .../core/client/operator/ReduceByKeyTest.java | 245 ++++++ .../core/client/operator/ReduceWindowTest.java | 129 ++++ .../core/client/operator/SumByKeyTest.java | 108 +++ .../core/client/operator/TopPerKeyTest.java | 130 ++++ .../euphoria/core/client/operator/UnionTest.java | 84 ++ .../core/client/type/TypePropagationAssert.java | 75 ++ .../euphoria/core/client/util/IOUtilsTest.java | 69 ++ .../core/docs/DocumentationExamplesTest.java | 850 +++++++++++++++++++++ .../core/testkit/AbstractOperatorTest.java | 118 +++ .../core/testkit/BroadcastHashJoinTest.java | 211 +++++ .../euphoria/core/testkit/CountByKeyTest.java | 143 ++++ .../euphoria/core/testkit/DistinctTest.java | 148 ++++ .../euphoria/core/testkit/FilterTest.java | 57 ++ .../euphoria/core/testkit/FlatMapTest.java | 113 +++ .../extensions/euphoria/core/testkit/JoinTest.java | 849 ++++++++++++++++++++ .../euphoria/core/testkit/MapElementsTest.java | 111 +++ .../euphoria/core/testkit/ReduceByKeyTest.java | 822 ++++++++++++++++++++ .../euphoria/core/testkit/ReduceWindowTest.java | 145 ++++ .../euphoria/core/testkit/SumByKeyTest.java | 66 ++ .../euphoria/core/testkit/TopPerKeyTest.java | 189 +++++ .../euphoria/core/testkit/UnionTest.java | 100 +++ .../euphoria/core/testkit/WindowingTest.java | 488 ++++++++++++ .../core/testkit/accumulators/LongCounter.java | 48 ++ .../core/testkit/accumulators/LongHistogram.java | 45 ++ .../core/testkit/accumulators/NanosecondTimer.java | 48 ++ .../accumulators/SingleJvmAccumulatorProvider.java | 154 ++++ .../testkit/accumulators/SnapshotProvider.java | 31 + .../core/testkit/accumulators/Snapshotable.java | 22 + .../core/testkit/accumulators/package-info.java | 20 + .../euphoria/core/testkit/package-info.java | 20 + .../core/translate/BeamMetricsTranslationTest.java | 174 +++++ .../euphoria/core/translate/EuphoriaTest.java | 103 +++ .../SingleJvmAccumulatorProviderTest.java | 78 ++ .../euphoria/core/translate/TestUtils.java | 32 + .../collector/SingleValueCollectorTest.java | 91 +++ .../beam/sdk/transforms/windowing/WindowDesc.java | 76 ++ sdks/java/extensions/kryo/build.gradle | 45 ++ .../apache/beam/sdk/extensions/kryo/KryoCoder.java | 287 +++++++ .../sdk/extensions/kryo/KryoCoderProvider.java | 167 ++++ .../beam/sdk/extensions/kryo/KryoOptions.java | 51 ++ .../beam/sdk/extensions/kryo/KryoRegistrar.java | 39 + .../apache/beam/sdk/extensions/kryo/KryoState.java | 125 +++ .../beam/sdk/extensions/kryo/package-info.java | 20 + .../sdk/extensions/kryo/KryoCoderProviderTest.java | 113 +++ .../beam/sdk/extensions/kryo/KryoCoderTest.java | 282 +++++++ .../beam/sdk/extensions/kryo/KryoStateTest.java | 52 ++ settings.gradle | 4 + website/src/documentation/sdks/euphoria.md | 573 +++++++++++++- 162 files changed, 18030 insertions(+), 39 deletions(-) create mode 100644 sdks/java/extensions/euphoria/README.md create mode 100644 sdks/java/extensions/euphoria/build.gradle create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/annotation/audience/Audience.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/annotation/audience/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/annotation/operator/Basic.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/annotation/operator/Derived.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/annotation/operator/Recommended.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/annotation/operator/StateComplexity.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/annotation/operator/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/annotation/stability/Experimental.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/annotation/stability/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/accumulators/Accumulator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/accumulators/AccumulatorProvider.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/accumulators/Counter.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/accumulators/Histogram.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/accumulators/Timer.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/accumulators/VoidAccumulatorProvider.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/accumulators/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/dataset/Dataset.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/dataset/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/BinaryFunction.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/BinaryFunctor.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/CombinableBinaryFunction.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/CombinableReduceFunction.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/CompositeUnaryFunction.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/Consumer.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/ExtractEventTime.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/ReduceFunction.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/ReduceFunctor.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/SameArgsBinaryFunction.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/Supplier.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/TernaryFunction.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/UnaryFunction.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/UnaryFunctionEnv.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/UnaryFunctor.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/UnaryPredicate.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/VoidFunction.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/functional/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/io/Collector.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/io/Context.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/io/Environment.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/io/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/lib/Euphoria.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/lib/Split.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/lib/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/AssignEventTime.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/CompositeOperator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/CountByKey.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/Distinct.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/Filter.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/FlatMap.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/FullJoin.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/Join.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/LeftJoin.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/MapElements.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/ReduceByKey.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/ReduceWindow.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/RightJoin.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/SumByKey.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/TopPerKey.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/Union.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/WindowBuilder.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/base/Builders.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/base/Operator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/base/OptionalMethodBuilder.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/base/ShuffleOperator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/base/WindowAware.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/base/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/ComputationHint.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/OutputHint.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/SizeHint.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/type/TypeAware.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/type/TypeAwareness.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/type/TypeUtils.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/type/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/util/Fold.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/util/Max.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/util/Sums.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/util/Triple.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/util/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/AbstractJoinTranslator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/BeamAccumulatorProvider.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/BroadcastHashJoinTranslator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/EuphoriaOptions.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/ExtractKey.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/FlatMapTranslator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/JoinTranslator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/LazyAccumulatorProvider.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/OperatorTransform.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/OperatorTranslator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/OperatorTranslators.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/ReduceByKeyTranslator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/SimpleTranslatorProvider.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/SingleValueContext.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/TranslatorProvider.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/UnionTranslator.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/collector/AdaptableCollector.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/collector/CollectorAdapter.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/collector/SingleValueCollector.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/collector/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/translate/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/util/IOUtils.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/util/InstanceUtils.java create mode 100644 sdks/java/extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/util/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/lib/SplitTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/CountByKeyTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/DistinctTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/FilterTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/FlatMapTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/JoinTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/MapElementsTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/OperatorTestUtils.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/ReduceByKeyTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/ReduceWindowTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/SumByKeyTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/TopPerKeyTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/UnionTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/type/TypePropagationAssert.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/util/IOUtilsTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/docs/DocumentationExamplesTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/AbstractOperatorTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/BroadcastHashJoinTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/CountByKeyTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/DistinctTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/FilterTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/FlatMapTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/JoinTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/MapElementsTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/ReduceByKeyTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/ReduceWindowTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/SumByKeyTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/TopPerKeyTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/UnionTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/WindowingTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/accumulators/LongCounter.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/accumulators/LongHistogram.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/accumulators/NanosecondTimer.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/accumulators/SingleJvmAccumulatorProvider.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/accumulators/SnapshotProvider.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/accumulators/Snapshotable.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/accumulators/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/testkit/package-info.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/translate/BeamMetricsTranslationTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/translate/EuphoriaTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/translate/SingleJvmAccumulatorProviderTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/translate/TestUtils.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/translate/collector/SingleValueCollectorTest.java create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/transforms/windowing/WindowDesc.java create mode 100644 sdks/java/extensions/kryo/build.gradle create mode 100644 sdks/java/extensions/kryo/src/main/java/org/apache/beam/sdk/extensions/kryo/KryoCoder.java create mode 100644 sdks/java/extensions/kryo/src/main/java/org/apache/beam/sdk/extensions/kryo/KryoCoderProvider.java create mode 100644 sdks/java/extensions/kryo/src/main/java/org/apache/beam/sdk/extensions/kryo/KryoOptions.java create mode 100644 sdks/java/extensions/kryo/src/main/java/org/apache/beam/sdk/extensions/kryo/KryoRegistrar.java create mode 100644 sdks/java/extensions/kryo/src/main/java/org/apache/beam/sdk/extensions/kryo/KryoState.java create mode 100644 sdks/java/extensions/kryo/src/main/java/org/apache/beam/sdk/extensions/kryo/package-info.java create mode 100644 sdks/java/extensions/kryo/src/test/java/org/apache/beam/sdk/extensions/kryo/KryoCoderProviderTest.java create mode 100644 sdks/java/extensions/kryo/src/test/java/org/apache/beam/sdk/extensions/kryo/KryoCoderTest.java create mode 100644 sdks/java/extensions/kryo/src/test/java/org/apache/beam/sdk/extensions/kryo/KryoStateTest.java